-- Logs begin at Wed 2022-08-10 10:25:18 UTC, end at Wed 2022-08-10 11:36:17 UTC. -- Aug 10 10:59:19.461453 dv-cinder-852724-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 10:59:19.469333 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 10:59:19 2022] *** Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: nodename: dv-cinder-852724-1-iscsi Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: machine: x86_64 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: clock source: unix Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: pcre jit disabled Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: detected number of CPU cores: 4 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: current working directory: / Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: detected binary path: /usr/bin/uwsgi-core Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: your processes number limit is 47680 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: your memory page size is 4096 bytes Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: detected max file descriptor number: 2048 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: lock engine: pthread robust mutexes Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: thunder lock: enabled Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 10:59:19.474598 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 10:59:19.512657 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: Python main interpreter initialized at 0x5613593559c0 Aug 10 10:59:19.512657 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: python threads support enabled Aug 10 10:59:19.512657 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: your server socket listen backlog is limited to 100 connections Aug 10 10:59:19.512657 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: your mercy for graceful operations on workers is 90 seconds Aug 10 10:59:19.513047 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 10:59:19.513125 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: *** Operational MODE: preforking *** Aug 10 10:59:19.513397 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: *** uWSGI is running in multiple interpreter mode *** Aug 10 10:59:19.513397 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: spawned uWSGI master process (pid: 80476) Aug 10 10:59:19.513600 dv-cinder-852724-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 10:59:19.518607 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: spawned uWSGI worker 1 (pid: 80490, cores: 1) Aug 10 10:59:19.518607 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: spawned uWSGI worker 2 (pid: 80491, cores: 1) Aug 10 10:59:19.518607 dv-cinder-852724-1-iscsi devstack@keystone.service[80476]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 10:59:24.994993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 10:59:24.994993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: from cryptography.utils import int_from_bytes Aug 10 10:59:24.995746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 10:59:24.995746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: from cryptography.utils import int_from_bytes Aug 10 10:59:24.997838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 10:59:24.997838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: from cryptography.utils import int_from_bytes Aug 10 10:59:24.998283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 10:59:24.998283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: from cryptography.utils import int_from_bytes Aug 10 10:59:25.045622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 10:59:25.045818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 10:59:25.046012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] command line args: [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 10:59:25.046285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 10:59:25.046464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ================================================================================ {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 10:59:25.046755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] admin_token = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.047004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] config_dir = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.047196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.047381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] config_source = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.047563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] debug = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.047764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.048910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.048910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.048910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 10:59:25.049170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 10:59:25.049288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] command line args: [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 10:59:25.049414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_config_append = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 10:59:25.049537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ================================================================================ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 10:59:25.049654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_dir = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_file = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] admin_token = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.049998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_dir = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] config_source = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] debug = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.050956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.051909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_config_append = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_dir = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.052998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.053939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] publish_errors = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.054963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] standard_threads = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.055959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] use_journal = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] use_json = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] use_stderr = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] use_syslog = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.056980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] publish_errors = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.057248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.057397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.057534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.057683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.057820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.057938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.058021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] standard_threads = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.058189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.058272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.058431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.058508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.058662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.058740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.058825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_journal = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.058995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.059072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_json = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.059168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_stderr = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.059301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.059377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] use_syslog = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.059578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 10:59:25.059654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.059837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.059914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.060078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.060205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.060339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.060516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.060592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.061975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.062970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.063873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.064973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.065980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.066932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.067961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.068905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069370 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.069855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.070957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.071902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.072945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.073863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.074992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.075961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.076979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.077938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.078967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.079946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.080979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.081886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.082872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.083987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.external = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.084988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.password = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] auth.token = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.external = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.085940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.password = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.086844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] auth.token = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.087968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.088947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.089951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.090992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.091803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.092525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.092616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.092779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.092861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.093842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.094859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.095985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.096961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.097990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.098984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.099964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.100910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.101877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.102898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.103991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.104946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.105975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.106980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.107133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.107211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.107365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.107442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.108936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.109967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.110943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.111922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.112906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.113997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.114819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.115937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.116976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.117863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.118933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] role.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] role.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.119981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.120879 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.121851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.122853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.123935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.124948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.125958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.126963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.127998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.128973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.129947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.130954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.131936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.132917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.connection = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.133900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.134885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.135838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.136966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80490) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 10:59:25.137439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.137922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.138186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.138440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.138714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.138977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 10:59:25.139150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80491) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 10:59:25.166471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.166741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.166973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.168360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.168614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.168850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.176766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.178745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.206045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.206274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.208188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.208397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.258660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.259289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.310522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.311176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.312039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.312278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.376922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.377102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.377328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.377742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.377832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.377994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.378912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.401480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.401663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.401800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.401911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.401994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.402904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.403055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.414327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.414919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.427742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.427977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.428958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.429095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80490) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.429177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.429333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.429548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80491) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 10:59:25.445845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.446060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 10:59:25.446142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.446277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 10:59:25.446408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.446639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.446723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 10:59:25.446845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 10:59:25.446927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.447943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.459535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.459819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.460933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.467356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.468615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.470949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.471241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.471531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.471798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.471880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.472972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.473999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.474275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.474563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.496635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.497949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.498126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.498443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.499504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.499810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.503377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.503670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.503956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.504793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.505077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.505378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.510561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.510878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.512143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.512451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.515510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.515802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.516088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.516385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.516677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.516976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.517183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.517272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.517473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.517552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.517775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.518067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.518360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.518672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.518955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.519226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.533989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.534277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.536024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.536325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.538385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.540449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.541548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.541853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.543651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.543970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.546276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.548457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.549450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.551768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.552655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.552961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.553251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.553549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.553839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.554131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.555049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.555351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.555659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.555965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.556244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.556531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.570024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.570324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.570614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.570887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.572870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.573159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.573430 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.573694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.578272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.579872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.580157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.581204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.582844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.582965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.583125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.583256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.583565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.583860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.584145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.585990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.586293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.586624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.586924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.587204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.596224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.596514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.596799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.597090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.597378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.597662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.597945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.599388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.599680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.599958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.600257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.600556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.600834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.601111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.614396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.617392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.617904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.620349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.620951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.621936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.623498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.623798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.624009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.624090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.625656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.627247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.627510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.628066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.630800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.632422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.634734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.635395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.635693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.635988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.636268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.636359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.639924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.640234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.640517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.640791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.644890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.645198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.645561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.649249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.649556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.649856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.652476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80490) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.652773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.653058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.653337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.653615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.653902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.654180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.654459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.655452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.655746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.656026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.656303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.656575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.656843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80490) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.657486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80491) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 10:59:25.657772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.658055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.658334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.658653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.658954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.659232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.659523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.659934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.660218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.660496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.660786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.661070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.661396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80491) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 10:59:25.689926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613593559c0 pid: 80490 (default app) Aug 10 10:59:25.693261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2774d9-8c30-4326-ae23-c2d024d6ad14 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:25.693557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2774d9-8c30-4326-ae23-c2d024d6ad14 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:25.693832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2774d9-8c30-4326-ae23-c2d024d6ad14 None None] PATH_INFO: `/v3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:25.694986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613593559c0 pid: 80491 (default app) Aug 10 10:59:25.698114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Wed Aug 10 10:59:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:33.336125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142437-c063-454f-b77a-15840a131e3a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:33.336698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142437-c063-454f-b77a-15840a131e3a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:33.336698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35142437-c063-454f-b77a-15840a131e3a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:33.341037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 10:59:33.346260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:33.347134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:33.347369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:33.417406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: INFO dbcounter [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] Registered counter for database keystone Aug 10 10:59:33.436984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG oslo_db.sqlalchemy.engines [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80490) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 10:59:33.443632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writer thread running {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 10:59:33.626194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG passlib.handlers.bcrypt [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80490) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 10:59:33.626529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG passlib.handlers.bcrypt [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80490) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 10:59:33.680888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:33.682470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:33.814245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 10:59:33.971994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35142437-c063-454f-b77a-15840a131e3a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:34.043730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: INFO dbcounter [None req-35142437-c063-454f-b77a-15840a131e3a None None] Registered counter for database keystone Aug 10 10:59:34.066547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG oslo_db.sqlalchemy.engines [None req-35142437-c063-454f-b77a-15840a131e3a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80491) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 10:59:34.072955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writer thread running {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 10:59:34.197865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35142437-c063-454f-b77a-15840a131e3a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:34.202313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:34.202997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:34.203298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:34.203585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:34.212075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:34.266641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 10:59:34.266965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a34d91-da30-44e2-a0aa-21d35226d6ed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:34.276902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 10 10:59:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 10:59:34.282959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:34.297579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e048f8-2bd3-4a49-bde8-dfae2762a8c6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:34.302626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:34.303729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:34.304008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:34.304280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] PATH_INFO: `/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:34.305699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] RBAC: Authorizing `identity:get_project(project_id=dd03b7da5c5c4a22928c3c3d2bce3325)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:34.355835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 10:59:34.356368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4065120-4e9e-4a3a-bc2a-a9d20bd20310 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:34.360655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Wed Aug 10 10:59:34 2022] GET /identity/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:36.337782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95782b-5c80-4392-9697-566329107416 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:36.338311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95782b-5c80-4392-9697-566329107416 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:36.338311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a95782b-5c80-4392-9697-566329107416 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:36.342399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:36.347765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:36.348185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:36.348344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:36.369239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:36.370828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:36.379636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 10:59:36.402750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95782b-5c80-4392-9697-566329107416 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:36.414914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a95782b-5c80-4392-9697-566329107416 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:36.419363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:36.419985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:36.420263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:36.420531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:36.428449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:36.430267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733322-0885-42ff-b43e-67f32428a35a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:36.439362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Wed Aug 10 10:59:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 10:59:36.445126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:36.454029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5d754-de95-44ff-a7db-e2c5d6cd88dc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:36.458405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:36.459652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:36.459938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:36.460217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:36.461551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:36.463407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c5f15-df25-4f14-a244-b5e8f00563f3 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:36.464905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Wed Aug 10 10:59:36 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:38.561474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb2221-f256-473c-97e7-03600173e8b2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.562008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb2221-f256-473c-97e7-03600173e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.562008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb2221-f256-473c-97e7-03600173e8b2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.563557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:38.568361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.568746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.569528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.587644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:38.588814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:38.592947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59f53-ea2c-480c-928d-1016775442c2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.593222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59f53-ea2c-480c-928d-1016775442c2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.593490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be59f53-ea2c-480c-928d-1016775442c2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.594806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 10:59:38.599676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.599958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.600228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.642517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG passlib.handlers.bcrypt [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80491) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 10:59:38.642975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG passlib.handlers.bcrypt [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80491) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 10:59:38.669852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:38.688255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:38.689809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:38.691754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:38.716656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:38.790621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 10:59:38.796541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5816e2c-948c-41b3-8d28-2ad819e22c5d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:38.811569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:38.829766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:38.830383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.830704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.830984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.906288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:38.907795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:38.912939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b934bf-8ddd-4b2a-a87b-d1be1df9bf89 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:38.916019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:38.931827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:38.953117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:38.954261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:38.954592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:38.954891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:38.967088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:38.975624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:38.976524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:38.977510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 8/14] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 10:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 10:59:38.982818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.009133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 7/15] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 10:59:39.014877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.083177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06d54c-8e7f-44f4-9b03-5796129f5661 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.119601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.120228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.120511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.120786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.121494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:39.121800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.123300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.139773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 9/16] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:39.145573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.173156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:39.215365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4a7d5-35f3-40e1-9a37-1331a2fdb94d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.236487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:39.245733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de82f8fd-58a5-491b-8da7-b51db93bf5ca None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.249117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.249751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.250032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.250308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.260963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.262422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.279217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.279840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 10:59:39.279961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:39.280792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 10:59:39.281608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 10:59:39.281608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 10:59:39.281608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:39.281768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.281768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.281889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 8/17] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 10:59:39.281961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.283316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.289583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 10/18] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:39 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:39.289716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.290815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.382024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:39.386525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587e30d-ac57-45df-b978-8765cf4e736b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.389178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90024e32-7ce9-4e60-8f79-bd68c2d49c48 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.420173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.420781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.421066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.421341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.421937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.422052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:39.422275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.422779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.423062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.423337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.423704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65da92-5c67-4dcf-bfcb-586d15e7bb6f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.424454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.425897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.427481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 11/19] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:39.433204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:39.446024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 9/20] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 10:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:39.536963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb33cd74-d173-448f-8f13-5ffaefebd321 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:39.572585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:39.573294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:39.573552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:39.573824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:39.574662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.domain.id=default, user.id=8727d28e1a63419baaa4c9f20a7bc777) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:39.574978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:39.576498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a0125-4cb7-4033-84c3-379afd848df5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:39.586941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 12/21] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 10 10:59:39 2022] GET /identity/v3/role_assignments?user.id=8727d28e1a63419baaa4c9f20a7bc777&scope.domain.id=default&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:41.861273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c386-7b5e-4412-8eea-c45de4348069 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:41.861799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c386-7b5e-4412-8eea-c45de4348069 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:41.861799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2c386-7b5e-4412-8eea-c45de4348069 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:41.863187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 10/22] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:41.870592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:41.871025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:41.871194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:41.892972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:41.894490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:41.920332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:41.943886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d49af-b6d9-4b9f-a3a8-17a4362b93c9 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:41.944215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d49af-b6d9-4b9f-a3a8-17a4362b93c9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:41.944499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d49af-b6d9-4b9f-a3a8-17a4362b93c9 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:41.946107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 11/23] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 10:59:41.956045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:41.956344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:41.956587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:41.985778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 13/24] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:41.999816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:42.001050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:42.032298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.045457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:42.085296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 12/25] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:42.175716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.188697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbd66f-7401-4e6a-a1ee-bd466cad2206 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.235759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.236400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.236684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.236961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.267983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:42.269452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7e897798-4eb6-492b-bab9-84f1487bb7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 10:59:42.302597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:42.303507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 10:59:42.304307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 10:59:42.304307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 10:59:42.304307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:42.306127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 14/26] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:42.333174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6054b6c-4c8a-4ae6-88bc-5896da7e0ca8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.371770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.372804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.373277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.373737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.388190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:42.394612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:42.395514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:42.396294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:42.398759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 13/27] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 10:59:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:42.400390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.456317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:42.478140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6d944-617f-4788-b348-f8175d0923d2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.478800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6d944-617f-4788-b348-f8175d0923d2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.479275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6d944-617f-4788-b348-f8175d0923d2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.480931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 15/28] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:42 2022] GET /identity => generated 270 bytes in 172 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:42.481862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6d944-617f-4788-b348-f8175d0923d2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.506981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:42.558661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e34ba-e60b-4831-9507-c7ef8d391086 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.602549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.604314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.604766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.605206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.605999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:42.606459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:42.608050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69179dd8-d0e0-44f3-9046-fbc36bd08877 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:42.623894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 14/29] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 10:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:42.625405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.625863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.626297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.641798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6d944-617f-4788-b348-f8175d0923d2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.650070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:42.651480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:42.687715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.688611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.689065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.689478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.690629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:42.691092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:42.693518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211cef8-1166-4bb8-8d45-b69fe77bec98 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:42.709439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 16/30] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:42.709439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfdd1d-1280-4782-8c8f-6acbeb0ec4df None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.709439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfdd1d-1280-4782-8c8f-6acbeb0ec4df None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.709439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfdd1d-1280-4782-8c8f-6acbeb0ec4df None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.709781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 17/31] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:42.712486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfdd1d-1280-4782-8c8f-6acbeb0ec4df None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.739231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 15/32] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:42.740715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:42.759789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:42.820640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:42.859468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751debbf-0cfb-45e1-9947-3029fdde4e8a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.875772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfdd1d-1280-4782-8c8f-6acbeb0ec4df None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:42.909350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.910728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.911237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.911665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.912298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:42.913005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.913290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:42.913466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.913837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.914605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:42.915996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a44e84-7a2b-4550-8c3e-cfbf20574aa1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:42.917070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:42.918215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 16/33] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:42 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:42.919671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:42.920120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:42.920549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:42.942845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:42.944564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:43.016680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 17/34] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:43.017775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.040258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.065476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 18/35] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 10:59:42 2022] POST /identity/v3/projects => generated 301 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:43.066611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.149554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc7c6d-0784-4b95-b5d4-875d2a16f270 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.181927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd4d04e-3a85-4c3a-b188-f622b02c9078 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.192329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.193337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.193814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.194230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.195212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.196904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.211554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.228985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.229637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.229924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.230201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.231467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.233065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce91abe-091b-410f-b086-b1c0c530d4f8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.235717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 19/36] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:43.237086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cfde1-4abb-41ba-af4b-428997898de5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.237368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cfde1-4abb-41ba-af4b-428997898de5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.237649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cfde1-4abb-41ba-af4b-428997898de5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.239040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 20/37] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:43.239725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cfde1-4abb-41ba-af4b-428997898de5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.337407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 18/38] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/roles => generated 225 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:43.337407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.370401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cfde1-4abb-41ba-af4b-428997898de5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.420442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.421230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.421604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.421994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.422964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.424941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4da570-692e-40cc-8df6-ed343313e080 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.428967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.444887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 21/39] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/roles => generated 219 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:43.446249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.446540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.446880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.476326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:43.478783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:43.479337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7bb90c-1864-4040-a019-cc833a12f389 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.526860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.527562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.527844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.528122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] PATH_INFO: `/v3/domains/default/users/8727d28e1a63419baaa4c9f20a7bc777/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.530473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8727d28e1a63419baaa4c9f20a7bc777, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.532135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fe8ae-efd5-433b-b230-c3d4a2b1a8fb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.554370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 19/40] 10.222.0.49 () {58 vars in 1361 bytes} [Wed Aug 10 10:59:43 2022] PUT /identity/v3/domains/default/users/8727d28e1a63419baaa4c9f20a7bc777/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:59:43.556389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc42a-6b39-4b3f-aa0d-89f69ccb7acb None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.556677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc42a-6b39-4b3f-aa0d-89f69ccb7acb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.557067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bc42a-6b39-4b3f-aa0d-89f69ccb7acb None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.558278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 20/41] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:43.562546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.564622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.564906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.565209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.587920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:43.589408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:43.600621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 22/42] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:43.626533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.673539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 21/43] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:43.696159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.763580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d17a76-ce40-4883-8a46-4f2351d0c3ca None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.769163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.803062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.803804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.804090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.804412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.805260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.806795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.808319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f1b1c-e7c0-401e-b41e-edf0b15f1c1a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:43.830613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 23/44] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/roles => generated 223 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:59:43.855969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:43.856646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:43.857026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:43.857369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:43.858701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:43.860268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f320-dc81-4c0e-a901-11258cae17bd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:43.862237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 22/45] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:43.868874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:43.916407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:43.982302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e13727d-df89-4626-b472-ad62f42ad481 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:44.020642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:44.021261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:44.021550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:44.021824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:44.022498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:44.023993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2d07d6-1e47-4994-a708-fed99c01eaec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:44.053427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 24/46] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 10:59:43 2022] POST /identity/v3/projects => generated 312 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:45.606786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb20fd-2dbd-47f2-a4e7-920be49876db None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:45.607321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb20fd-2dbd-47f2-a4e7-920be49876db None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:45.607321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb20fd-2dbd-47f2-a4e7-920be49876db None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:45.608691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 23/47] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:45.613965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:45.614347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:45.614662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:45.633358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:45.634528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:45.700218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 25/48] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:45.722781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb20fd-2dbd-47f2-a4e7-920be49876db None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:45.821474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb20fd-2dbd-47f2-a4e7-920be49876db None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:45.850322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:45.857247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:45.857821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:45.858100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:45.858372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:45.868679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:45.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:45.874928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:45.875710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 24/49] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 10:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:45.881291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:45.905355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:45.978479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f389e5-2a4a-4882-8c88-42d8b2b34df0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.010999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.011603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.011888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.012166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.012841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:46.013148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.014820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28c516-68d8-4652-ac39-c0e25c258066 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.022686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 26/50] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:46.028680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:46.040070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f6cd-228b-446a-9a1a-b0bc80032fe2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.040358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f6cd-228b-446a-9a1a-b0bc80032fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.040638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e379f6cd-228b-446a-9a1a-b0bc80032fe2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.042509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 27/51] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 10:59:46.047389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.047671 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.047945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.065114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:46.066301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:46.128260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd89b23-9373-4136-b645-fe0d94c5318b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.132352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 28/52] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 10:59:46.156081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:46.161618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.162244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.162526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.162832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.163957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.165429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.167181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 25/53] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:46 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:46.170295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:46.172434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:46.228393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:46.251962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7183f-dc16-465b-ad0c-b0f847583e50 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.269525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9056a83a-9fd5-4fc3-a24b-64f722d28185 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.285710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.286332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.286644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.286922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.288041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.289474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.291056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 29/54] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:46.296716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:46.302747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.303373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.303654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.303924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.305036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.306496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.308088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 26/55] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:46.313572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:46.374730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:46.393268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d14929c-b7a9-43f3-a431-161fb7eaddce None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.409357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114a22a-5b42-4cb7-8170-fe210e070aaa None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:46.426234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.426910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.427191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.427484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.428124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.429534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4dd16-35eb-4ead-843e-5aaa21d65a9c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.434491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:46.443594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:46.444190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:46.444472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:46.444745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:46.445467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.domain.id=default, user.id=8727d28e1a63419baaa4c9f20a7bc777) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:46.445774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:46.447249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec18c3a-8976-4ff0-a6d2-caf3510b6b34 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:46.455313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 30/56] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 10 10:59:46 2022] POST /identity/v3/projects => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:59:46.455975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 27/57] 10.222.0.49 () {56 vars in 1322 bytes} [Wed Aug 10 10:59:46 2022] GET /identity/v3/role_assignments?user.id=8727d28e1a63419baaa4c9f20a7bc777&scope.domain.id=default&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 10:59:48.458310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13753ab3-91f1-4594-b833-4a2f51ade06f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:48.458877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13753ab3-91f1-4594-b833-4a2f51ade06f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:48.458877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13753ab3-91f1-4594-b833-4a2f51ade06f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:48.460209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:48.465808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:48.466166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:48.466363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:48.485194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:48.486374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:48.550368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:48.574396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13753ab3-91f1-4594-b833-4a2f51ade06f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:48.670492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13753ab3-91f1-4594-b833-4a2f51ade06f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:48.701883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:48.702487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:48.702793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:48.703067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:48.704173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:48.705616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e721b7-0e6a-4862-abf3-281339528310 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:48.707163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:48.712659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:48.807570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb24d70-e153-4be1-b4ff-3039b15bf07e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:48.826346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:48.840591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:48.841183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:48.841467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:48.841744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:48.842396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:48.843864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78436460-80b1-4c93-b319-f253af63f64b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:48.870849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 10:59:48 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 10:59:51.821200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a7cc4-dd37-405f-b8c1-921651fb2165 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:51.821701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a7cc4-dd37-405f-b8c1-921651fb2165 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:51.821775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a7cc4-dd37-405f-b8c1-921651fb2165 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:51.823168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:51.828407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693dc5-f83c-4e2a-9674-397893cf7d33 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:51.828817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693dc5-f83c-4e2a-9674-397893cf7d33 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:51.828983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693dc5-f83c-4e2a-9674-397893cf7d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:51.851616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-97693dc5-f83c-4e2a-9674-397893cf7d33 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:51.852808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-97693dc5-f83c-4e2a-9674-397893cf7d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:51.953062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 30/63] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 10:59:52.002614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a7cc4-dd37-405f-b8c1-921651fb2165 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.005575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.130076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550e8a7-9d95-4ae4-ba54-e49923f05c50 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.130397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550e8a7-9d95-4ae4-ba54-e49923f05c50 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.130586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550e8a7-9d95-4ae4-ba54-e49923f05c50 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.131911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 31/64] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:52.132613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a7cc4-dd37-405f-b8c1-921651fb2165 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.143941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155c09a-6cd8-445b-8fc5-c4ebc13d8468 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.144224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155c09a-6cd8-445b-8fc5-c4ebc13d8468 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.144501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155c09a-6cd8-445b-8fc5-c4ebc13d8468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.173799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0155c09a-6cd8-445b-8fc5-c4ebc13d8468 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:52.175340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0155c09a-6cd8-445b-8fc5-c4ebc13d8468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:52.178249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.178912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.179191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.179461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.196397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ccf4bb8e-016d-47b8-8c4f-6804c2c19c5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:52.203193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:52.204086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:52.205151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 34/65] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:52.205939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac2f31-c4fb-4b21-b5b2-caa69513a0cc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.205939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac2f31-c4fb-4b21-b5b2-caa69513a0cc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.206082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac2f31-c4fb-4b21-b5b2-caa69513a0cc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.207362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 35/66] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:52.214067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac2f31-c4fb-4b21-b5b2-caa69513a0cc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.266332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 32/67] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:52.268165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.268627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.269036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.286988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:52.289170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:52.297989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.329340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.350625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac2f31-c4fb-4b21-b5b2-caa69513a0cc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.379259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 33/68] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:52.386662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.399593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.400215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.400498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.400772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.401447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:52.401752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:52.405460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:52.419500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 36/69] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:52.420447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.507896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391d71c-1c25-4a6a-890b-c45d17f87af4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.527526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74334f3-1b53-4b77-a092-a87578c345bd None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.546180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.547107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.547401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.547653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.553725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.558484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:52.563811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:52.564673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:52.565447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 34/70] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:52.566078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.571811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.572698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.573160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.573435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.584309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0cf5e6c5-ee01-4d0e-9099-5219f0d35fe6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:52.589011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:52.589866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:52.591683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 37/71] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:52.592609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ed5a-63f5-4a43-aca2-4a51ebaabbd4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.592609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ed5a-63f5-4a43-aca2-4a51ebaabbd4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.592894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7ed5a-63f5-4a43-aca2-4a51ebaabbd4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.598649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 38/72] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:52.598649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7ed5a-63f5-4a43-aca2-4a51ebaabbd4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.605059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.680288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c8c41-b20d-40f1-987a-3fae94a20f5f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.703373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7ed5a-63f5-4a43-aca2-4a51ebaabbd4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.713632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.714258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.714543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.714851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.727502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:52.728969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:52.730900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 35/73] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:52.731536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.738096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.738816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.739162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.739499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.740227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:52.740592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:52.742020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a50aa0-6e78-47cb-aafb-449966f44889 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:52.750631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 39/74] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:52.751818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291129-7b2b-44da-bba5-70cc79c08c60 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.752106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291129-7b2b-44da-bba5-70cc79c08c60 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.752378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df291129-7b2b-44da-bba5-70cc79c08c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.769668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-df291129-7b2b-44da-bba5-70cc79c08c60 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:52.770862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df291129-7b2b-44da-bba5-70cc79c08c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:52.812363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.821453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:52.837502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 40/75] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:52.838052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cca80-813f-467a-98d3-19ab9ae27c30 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.838741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391613e-c8ae-4677-b356-2f0540f8ef23 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.839028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391613e-c8ae-4677-b356-2f0540f8ef23 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.839299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2391613e-c8ae-4677-b356-2f0540f8ef23 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.841633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 41/76] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:52.842025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391613e-c8ae-4677-b356-2f0540f8ef23 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.876061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.876669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.876950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.877227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:52.877891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:52.878195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:52.879585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:52.890873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 36/77] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:52.890873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:52.955038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2391613e-c8ae-4677-b356-2f0540f8ef23 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.990846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:52.991390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a245f-2144-4443-a7ff-72a650a46cc4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:52.991510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:52.991713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:52.991983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.025039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.025668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.025954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.026231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.027409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.028892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f6fe4-a775-4bce-b540-4b04acb56cb8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.030466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 37/78] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:53.035404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.035404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.035404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.036578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.050694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:53.052054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:53.059508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.061553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.061850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.124471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 38/79] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:53.137237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.161096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 42/80] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 10:59:53.161810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.224573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20505c-b083-4869-995f-02c55c935a41 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.235112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.258002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.258177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b6cd9-5554-4a99-80c8-2da74bdab69c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.258647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.258930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.259197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.269322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:53.273910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:53.274985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 39/81] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:53.282004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.282813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.299441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.300306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.300611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.300923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.302162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.303725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.309612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 43/82] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:53.309612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.377263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cfc9de-c2a2-474e-9d33-5f704b1166d8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.403983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072bc38-be91-4f64-91a4-64c99d509975 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.410026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.410648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.410932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.411205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.437088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.437705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.437984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.438255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.439378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.440822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.442609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 44/83] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:53.443135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.456904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.481029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.483028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.485614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.539588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c8614-ebc4-4583-afda-c84d3b0fc603 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.571996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.572753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.573032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.573325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.574440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.576414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.578170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 45/84] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:53.578855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.581789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 40/85] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 10:59:53.582788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.660595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.678968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d4178-8bc7-47df-9347-d124f8c86521 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.681210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a9bb-8981-46e6-b2fb-bb6326cd9275 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.684371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.712262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.712862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.713148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.713423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.714087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:53.714391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.715803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.717192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.717799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.718086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.718364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.724383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 46/86] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:53.725149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.788516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.789916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.823072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca4edc-eba0-4473-bd40-cdef408b25e7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:53.861029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:53.862258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:53.862732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:53.863145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:53.863923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:53.865480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:53.886509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 41/87] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 10:59:53.886509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.897968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 47/88] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 10:59:53 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:53.897968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:53.900551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:53.902055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.007224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9dca80-24ab-4b1e-bafa-cac1303ad081 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.013243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b483117-5cf9-447d-9de5-6dca4bdf479b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.046987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.047560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.047841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.048826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.049304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.050844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.052620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 48/89] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.053381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.054795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.055417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.056039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.056039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.056710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:54.057029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.058755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.070868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 42/90] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 10:59:53 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.071862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.140245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.158085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.195063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec2a54-f95e-4e61-97df-f1fa2dcb93ed None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.199785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72fa59-32ef-4e63-a1d1-433b5c4a2989 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.249076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.249276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.249715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.249996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.250272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.250824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.251289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.251745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.251839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.253175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.253277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.255118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 49/91] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.255491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.255689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.257681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 43/92] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.265084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.385356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5ab57-9076-42ec-b7f6-266f6c0f1293 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.386115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830eed4-9ffc-4e8d-bf0d-2cfd7640dc35 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.422188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.422916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.423192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.423473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.424102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.424215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=8727d28e1a63419baaa4c9f20a7bc777) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:54.424518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.425206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.425680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.425960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.426323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.427340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:54.427845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.428302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.430702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.432103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 50/93] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/role_assignments?user.id=8727d28e1a63419baaa4c9f20a7bc777&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.432988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.434854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.441068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 44/94] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:54.591277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf33ec-6573-4b66-b943-0d6376078933 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:54.646161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:54.647187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:54.647665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:54.648130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:54.732120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:54.737765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:54.739570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:54.871786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 51/95] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 10:59:54.877464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:54.998872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef7934-82bc-452a-a17f-af1d7ebf07f1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:55.046312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:55.047030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:55.047316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:55.047595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] PATH_INFO: `/v3/projects/cb2fe3eaa1ad46018859e190dc89800a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:55.049555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] RBAC: Authorizing `identity:get_project(project_id=cb2fe3eaa1ad46018859e190dc89800a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:55.051143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe343bf-f38b-4f17-9f5e-38f315a096f3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:55.052987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 45/96] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:54 2022] GET /identity/v3/projects/cb2fe3eaa1ad46018859e190dc89800a => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:55.059075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:55.186287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:55.191963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a771e61f-7ced-4208-ab0b-86e31f40eff8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:55.235286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:55.235908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:55.236189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:55.236467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:55.237213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=cb2fe3eaa1ad46018859e190dc89800a, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:55.237525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:55.239065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedba899-8f44-4c3c-91e0-6074a770af40 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:55.245183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 52/97] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 10:59:55 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=cb2fe3eaa1ad46018859e190dc89800a&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:57.011660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e29d-57a9-431f-bb89-ec28f8caf81e None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.012153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e29d-57a9-431f-bb89-ec28f8caf81e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.012265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e29d-57a9-431f-bb89-ec28f8caf81e None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.013570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 46/98] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:57.019036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907904ff-59a3-45e9-a4c4-1156f4b561a3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.019425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907904ff-59a3-45e9-a4c4-1156f4b561a3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.019609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907904ff-59a3-45e9-a4c4-1156f4b561a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.037586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f34d-ed2f-43d8-86a3-afdff5964494 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.038172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f34d-ed2f-43d8-86a3-afdff5964494 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.038720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032f34d-ed2f-43d8-86a3-afdff5964494 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.040372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 47/99] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 10:59:57.045682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.046778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.047255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.047625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-907904ff-59a3-45e9-a4c4-1156f4b561a3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:57.048816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-907904ff-59a3-45e9-a4c4-1156f4b561a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:57.065862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:57.067401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:57.149099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.151762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 53/100] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:57.153076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade8d35-a8d2-48fd-9466-71af9c63b1ab None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.153384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade8d35-a8d2-48fd-9466-71af9c63b1ab None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.153662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade8d35-a8d2-48fd-9466-71af9c63b1ab None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.158313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 54/101] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:57.159651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 48/102] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 10:59:57.164717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.165001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.165280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.183589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:57.184773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:57.186645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.325352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 55/103] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:57.326220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.338252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17684875-ddb4-497d-9b91-5355f680fa85 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.382936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.400660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.401244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.401527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.401800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.426006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:57.430815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:57.431671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 10:59:57.432439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:57.433915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 49/104] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 10:59:57.435101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.490702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc7492-96e9-4f01-aa91-fb9d550f9ffd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.505647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.529389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.530011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.530319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.530616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.540808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9fe81ad7-a862-4000-999f-d061081fbf79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:57.545570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:57.547117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:57.548106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 56/105] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:57.549552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eadd3-af1d-4c39-a159-34c420c7314c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.549874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eadd3-af1d-4c39-a159-34c420c7314c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.550178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451eadd3-af1d-4c39-a159-34c420c7314c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.554632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 57/106] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:57.554750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3352e08-e088-4f63-80ed-0a385c8dc8f8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.554848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82ecc-d30f-4ae2-9110-28681f5ea782 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.554848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82ecc-d30f-4ae2-9110-28681f5ea782 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.554848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb82ecc-d30f-4ae2-9110-28681f5ea782 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.554848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 58/107] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:57.555393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82ecc-d30f-4ae2-9110-28681f5ea782 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.594064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.594721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.595007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.595283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.596389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:57.597846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:57.602622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 50/108] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:57.602622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.650773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.670375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb82ecc-d30f-4ae2-9110-28681f5ea782 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.705931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7912dcf-3ac4-4201-bab9-25760524936e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.706269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.706904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.707216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.707560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.708338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:57.708642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:57.710020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60674910-5753-4942-bf79-84bb9c026fbe None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:57.723411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 59/109] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:57.723411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.723411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.723411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.737699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.740810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:57.742083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:57.747338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.747959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.748248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.748531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.749249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:57.749563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:57.750995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6539b-3614-4fe7-a94d-1039cd4c4356 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:57.759524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 51/110] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:57.765166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.765463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.765748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.783551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:57.784758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:57.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 60/111] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:57.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.866374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 52/112] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:57.867308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:57.913294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:57.941241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae03c95-ff76-41bb-a8a9-1577568643d3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.977677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0a1a2-3006-4933-8250-aa30c11313da None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:57.979138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:57.979773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:57.980662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:57.980662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:57.980969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:57.982399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:57.995614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.010267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 61/113] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 10:59:57 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 10:59:58.011029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.022192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.023456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.023880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.024322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.025775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.027474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.029341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 53/114] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:57 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.030299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.165505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f98c7-e43d-4e5b-a382-70d17df5a62d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.182130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af4849-d426-4782-9059-65042037d5c2 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.204164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.219789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.219789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.220052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.220374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.221484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.224830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.224830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 62/115] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.225752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.227735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.228368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.228655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.228935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.241777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:58.247063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:58.247938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 10:59:58.248744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 54/116] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:58.250014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.300079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.350874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469495dc-e362-4a58-b02a-755af91bb79f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.359882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b2869-b722-4174-8103-210fd969ff54 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.386843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.387477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.387757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.388029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.395596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.396401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.396848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.397265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.398638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:58.398810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.400544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.402875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 55/117] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:58.403828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:58.404772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:58.405684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 63/118] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:58.406396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.406595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.472088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.518670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6785f506-fe6b-4ffa-ab48-5b99279361c4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.524364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab31dc2-9227-441e-af7d-d03b0e0acaa8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.539636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.555554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.556169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.556450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.556723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.557393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:58.557696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.559119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd5a65-cd36-40f1-9037-f497bb3ae0d6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.564549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.565374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.565858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.566267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.567231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 64/119] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.571924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0af84e-d9f1-47b1-aeb0-783539ab68dc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.571924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0af84e-d9f1-47b1-aeb0-783539ab68dc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.571924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0af84e-d9f1-47b1-aeb0-783539ab68dc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.571924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 65/120] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 10:59:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 10:59:58.571924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0af84e-d9f1-47b1-aeb0-783539ab68dc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.572659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.574287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.580582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 56/121] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.580582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.678828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860fc22-e64f-4c46-9d13-422a0fc9c5bb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.683725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0af84e-d9f1-47b1-aeb0-783539ab68dc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.702272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.713309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.713966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.714248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.714527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/26e85b5ffec84d34acc86041db6d5307/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.717387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, user_id=26e85b5ffec84d34acc86041db6d5307, role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.719682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.723629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.724391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.724885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.725180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.726169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:58.726431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.728092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3428-6438-4ee8-9df8-f2f2b228f158 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.742622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 57/122] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 10:59:58 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/26e85b5ffec84d34acc86041db6d5307/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:59:58.742622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.751081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 66/123] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.752567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.752859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.753795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.782815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 10:59:58.784773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 10:59:58.787422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:58.869731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff3a6f-c081-43bb-9a13-59d02fee3b61 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:58.889882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 67/124] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 10:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 10:59:58.892126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:58.911515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:58.912183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:58.912468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:58.912750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] PATH_INFO: `/v3/users/098934999774498399ba28290b19d6cc` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:58.928863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] RBAC: Authorizing `identity:get_user(user_id=098934999774498399ba28290b19d6cc)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:58.930344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:58.932211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 58/125] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/users/098934999774498399ba28290b19d6cc => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:58.935385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.030059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c1ef5-c50a-4d3c-a4b0-885c2eef225f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.043086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.068514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696c94a-f304-42a5-ad64-79265982efa0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.070140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.078762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.079654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.080079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.080495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/26e85b5ffec84d34acc86041db6d5307/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.082601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, user_id=26e85b5ffec84d34acc86041db6d5307, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.084175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.102447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 68/126] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 10:59:58 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/26e85b5ffec84d34acc86041db6d5307/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:59:59.103736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.110001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.110781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.111117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.111398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.112692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.114260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.116239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 59/127] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:58 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:59.117101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.222181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e4a1e-5a35-47ff-ae12-f52c9713d867 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.230213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5582ba-b5ea-42dc-a6f0-759239bf1bb2 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.261452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.262530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.263066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.263531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.267919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.268537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.268823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.269093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.277458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 10:59:59.284109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 10:59:59.285067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 10:59:59.285836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 10:59:59.288334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 69/128] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 10:59:59.289384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.308275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.330955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.349109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.349884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.433629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8d2e5-ad1a-450a-b8a1-e965fc1bb329 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.469903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 60/129] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 10:59:59.473593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.474423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.474874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.475282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.475388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.476625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.478327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.480204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 70/130] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:59.481077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.579014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.637937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 10:59:59.646332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146e1b0-2c7c-4010-ae72-35a6eb3c105e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.648044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee95969-a334-476a-a002-70bc86689d00 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.683811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.684677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.685094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.685503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.686842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.688474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.694198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 71/131] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:59.695373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.732170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.732896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.733075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.733348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.734022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 10:59:59.735598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.736305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.758078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 61/132] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 10:59:59.758776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.802207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377294-4a1c-4122-b723-19a27cac1fc8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 10:59:59.836539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 10:59:59.837537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 10:59:59.837966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 10:59:59.838373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/8727d28e1a63419baaa4c9f20a7bc777/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 10:59:59.840464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, user_id=8727d28e1a63419baaa4c9f20a7bc777, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 10:59:59.842015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 10:59:59.865615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 72/133] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 10:59:59 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/8727d28e1a63419baaa4c9f20a7bc777/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 10:59:59.866764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 10:59:59.899116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:00.015080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff36d10-0c25-4134-ba9a-c9a1c8939e18 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.075091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.076154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.076630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.077103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.080704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.082468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.084518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 73/134] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:00.090099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:00.111777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95276321-3bf1-4cf5-999a-18fbd4bb7b86 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.158937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:00.166461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.167207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.167704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.167991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.168748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=098934999774498399ba28290b19d6cc) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:00.169166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.170838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.183928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 62/135] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 10:59:59 2022] GET /identity/v3/role_assignments?user.id=098934999774498399ba28290b19d6cc&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 259 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:00.223392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:00.227218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d3234-874f-4333-9f96-3fbf882993e4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.274322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.275006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.275287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.275555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] PATH_INFO: `/v3/projects/cb2fe3eaa1ad46018859e190dc89800a` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.276736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] RBAC: Authorizing `identity:get_project(project_id=cb2fe3eaa1ad46018859e190dc89800a)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.278170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0339fd8-880e-4ba2-a8fc-801513078de7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.280182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 74/136] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:00 2022] GET /identity/v3/projects/cb2fe3eaa1ad46018859e190dc89800a => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:00.286589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:00.401544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa9e7-3c7a-4f44-8804-f4da04b1a73c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.401834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa9e7-3c7a-4f44-8804-f4da04b1a73c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.402011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aa9e7-3c7a-4f44-8804-f4da04b1a73c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.406374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 75/137] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:00.410619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.411146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.411606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.436944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:00.439289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:00.448411 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1113f9-1ae0-405a-b058-941b0cf758a1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.497007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.498000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.498472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.498976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] PATH_INFO: `/v3/projects/cb2fe3eaa1ad46018859e190dc89800a/users/26e85b5ffec84d34acc86041db6d5307/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.502720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb2fe3eaa1ad46018859e190dc89800a, user_id=26e85b5ffec84d34acc86041db6d5307, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.506002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.528703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 63/138] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:00 2022] PUT /identity/v3/projects/cb2fe3eaa1ad46018859e190dc89800a/users/26e85b5ffec84d34acc86041db6d5307/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:00.549675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 76/139] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:00.598542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:00.673280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:00.735012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94b645-555d-4811-ac2e-a6ed6f027976 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.773807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.774453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.774815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.775093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.776227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.777761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0fb5d0-d9da-4977-80a2-81ebc2ba9d61 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.779366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 64/140] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:00.785153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:00.892505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:00.902288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46cd1f-38e4-427f-86b8-bdc1c9091967 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:00.943496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:00.944116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:00.944398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:00.944672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:00.945391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:00.946868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ba477-ffcc-4e0a-bd6f-51821ae87899 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:00.983655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 77/141] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:00 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:01.676501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e138d-5635-4425-afc9-3807ed2abbaf None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:01.676999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e138d-5635-4425-afc9-3807ed2abbaf None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:01.677073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e138d-5635-4425-afc9-3807ed2abbaf None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:01.678528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 65/142] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:01.683963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:01.684402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:01.684573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:01.706667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:01.707843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:01.796216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 78/143] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:01.830049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e138d-5635-4425-afc9-3807ed2abbaf None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:01.950109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e138d-5635-4425-afc9-3807ed2abbaf None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:01.977678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:01.991073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:01.991710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:01.991996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:01.992274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.003684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:02.008412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:02.009253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:02.010017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 66/144] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:02.017130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.100837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:02.131737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f392da0-2e7e-42e0-bdbd-88b2242f1e9c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:02.171059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:02.171755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.172042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.172315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.173530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:02.173893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:02.175301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:02.185713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 79/145] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:02.191745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.307421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d2567-7d66-4ddf-b099-282117ff0b2f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:02.346783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:02.347430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.347713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.348045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.349254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:02.350777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266bc8a-a8f7-4def-be79-0cd46ef34d5b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:02.352663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 67/146] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:02.357845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.429537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2dd-299f-4a31-9e01-8443d89cd876 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.429841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2dd-299f-4a31-9e01-8443d89cd876 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.430103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe2dd-299f-4a31-9e01-8443d89cd876 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.431793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 68/147] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:02.436303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.436666 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.437040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.466839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:02.468151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:02.497356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab51819-9164-4d13-8597-d9f427cb13dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:02.546545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:02.547203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.547804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.548569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.551584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:02.571033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:02.592230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 69/148] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:02.646887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.682120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:02.683605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:02.783453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebc38e-ea67-4a17-be41-64d5bfb64bce None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:02.855786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:02.856361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:02.856647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:02.856921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:02.886014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 80/149] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:02.886567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:02.895313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:02.896461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:02.897248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 70/150] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:02.898667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.905302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:02.975762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.002489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.118860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d106a5-8261-43bc-85b8-a64dcf7ac77f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:03.128741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa0bf7-7737-48b0-9d21-5675fc9081dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:03.183327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:03.184810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.185402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.185831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.187006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:03.187457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:03.189556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fa7c5-2a0c-478b-b81e-ca397537f8c5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:03.201675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 71/151] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:03.203163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848909a-ff7f-4455-8305-43feeabd7d41 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.204047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848909a-ff7f-4455-8305-43feeabd7d41 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.205174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848909a-ff7f-4455-8305-43feeabd7d41 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.206863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 72/152] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:03.212155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466da4-3996-40ed-bc41-1947e3ac51b8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.212611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466da4-3996-40ed-bc41-1947e3ac51b8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.213031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466da4-3996-40ed-bc41-1947e3ac51b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.218714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:03.219357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.219637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.219957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.223375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:03.227025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:03.231631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 81/153] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:02 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:03.232542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:03.244928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-fb466da4-3996-40ed-bc41-1947e3ac51b8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:03.246627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fb466da4-3996-40ed-bc41-1947e3ac51b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:03.354041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6aed6-0746-476a-8e88-0fb442cba9bc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:03.362665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.392885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:03.393803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.394240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.394694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.395993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:03.397887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:03.399776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 82/154] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:03 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:03.400690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:03.427877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 73/155] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:03.429209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39900beb-6abe-4d9a-9725-eec41a843179 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.429506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39900beb-6abe-4d9a-9725-eec41a843179 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.429784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39900beb-6abe-4d9a-9725-eec41a843179 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.431289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 74/156] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:03.432002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39900beb-6abe-4d9a-9725-eec41a843179 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:03.504300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.510761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfa579-5499-4a39-9559-64e06c28e15e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:03.547552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:03.549099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.549597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.550070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.551126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:03.551701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:03.553389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf33c3fc-f74c-491e-9414-f2c4cce538ca None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:03.560042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 83/157] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:03 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:03.561526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.562120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.562621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.582225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:03.583854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:03.643175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.659972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39900beb-6abe-4d9a-9725-eec41a843179 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:03.719398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 84/158] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:03.720150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:03.752573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:03.753214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:03.753487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:03.753798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:03.871043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:03.873201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:03.921997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:03.995548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 75/159] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:03.996935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.023537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d125ea48-7d70-4498-b443-94d8afcf5c19 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.092933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.093676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.093958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.094229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.115785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:04.121833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:04.122872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:04.123689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 85/160] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:04.124412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.200995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2247988d-cc84-4b92-a060-a185ea754b66 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.202859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:04.278934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:04.295286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.295940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.296223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.296498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.324214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59fde67-14f0-4a5b-b0a6-96df870c08c0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.324523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:04.329219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:04.330206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:04.331055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 76/161] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:04.331715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.366109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.366773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.367054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.367325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.368593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:04.371009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:04.374258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 86/162] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:04.375123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.480931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4144bf-ea0a-45e9-b308-14ba9d8c92b3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.524184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.524801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.525086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.525362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.528301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:04.529764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:04.531221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:04.552593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 77/163] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:04.554281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.632446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:04.647409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2b2b4-5629-4a97-ab5e-d0279e250005 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.695288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:04.696400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.697080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.697156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.698815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.699568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:04.699885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:04.703132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b721fee-64b8-451b-99d5-afeaba0bcb16 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:04.722916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 87/164] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:04.726422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2856c2-c314-4247-8987-7884cc53ad6f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.726737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2856c2-c314-4247-8987-7884cc53ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.727016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2856c2-c314-4247-8987-7884cc53ad6f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.733319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 88/165] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:04.734118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2856c2-c314-4247-8987-7884cc53ad6f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.787087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85e0161-8766-4bb6-9d14-fe11d7cb107f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.838992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.839979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.840470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.840947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.841917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:04.842433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:04.844135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:04.856283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 78/166] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:04.857392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:04.924810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2856c2-c314-4247-8987-7884cc53ad6f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.973400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ffb84a-aa87-43e7-bcac-6e6b2bfdc530 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:04.981831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:04.982838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.982838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.983159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] PATH_INFO: `/v3/users/098934999774498399ba28290b19d6cc` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.984980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] RBAC: Authorizing `identity:get_user(user_id=098934999774498399ba28290b19d6cc)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:04.986994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b696e9-47f0-4767-84f4-fdb662f76009 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:04.988908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 89/167] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/users/098934999774498399ba28290b19d6cc => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:04.990366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:04.990674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:04.991007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:04.996874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.018296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.019378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.019874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.020371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] PATH_INFO: `/v3/users/8727d28e1a63419baaa4c9f20a7bc777` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.022146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] RBAC: Authorizing `identity:get_user(user_id=8727d28e1a63419baaa4c9f20a7bc777)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.023976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ee5ecf-37ad-4b33-919b-0fd12bcfd5fa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.025837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 79/168] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:04 2022] GET /identity/v3/users/8727d28e1a63419baaa4c9f20a7bc777 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:05.027307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f6a27-5687-4be1-846c-0be80a40b823 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.027795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f6a27-5687-4be1-846c-0be80a40b823 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.028286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f6a27-5687-4be1-846c-0be80a40b823 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.030105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 80/169] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:05.030309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:05.031631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f6a27-5687-4be1-846c-0be80a40b823 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.031734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:05.060282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.151573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 90/170] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:05.152904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.183389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f6a27-5687-4be1-846c-0be80a40b823 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.228918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.229871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.230406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.230889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.232684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.235570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fee73-2d8e-48d1-aad5-864674fc50bc None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.238614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 81/171] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:05.240261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.240717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.241163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.265360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:05.266784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:05.326268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e47aa-c30d-4c18-8f54-6925a6b8028d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.328550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.354879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 82/172] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:05.357951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.392500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.394361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.394944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.395644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.428571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.497364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.498893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.501390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d368c90-318e-4489-b763-02a3e7b084ff None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.537822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.538560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.538868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.539141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.550386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:05.555305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:05.556163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:05.556948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 83/173] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:05.559416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.609076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 91/174] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:05.609918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.615165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.672232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.672946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bd5a1-6384-465e-b60d-cd7bce2831d3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.707582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.708265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.708549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.708824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/098934999774498399ba28290b19d6cc/roles/332f5677bf6242369ec55ecd88ac6ad7` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.710784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, user_id=098934999774498399ba28290b19d6cc, role_id=332f5677bf6242369ec55ecd88ac6ad7)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.712267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e6577-5f9d-47f3-bf4f-d8ae524fba80 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.716355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683363c-ada5-4dd5-befa-dc343b003d13 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.724318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 84/175] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:05 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/users/098934999774498399ba28290b19d6cc/roles/332f5677bf6242369ec55ecd88ac6ad7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:05.725481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0799a85-61f6-4f27-81a2-a1d6954e1e60 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.725767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0799a85-61f6-4f27-81a2-a1d6954e1e60 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.726044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0799a85-61f6-4f27-81a2-a1d6954e1e60 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.727354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 85/176] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:05.728076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0799a85-61f6-4f27-81a2-a1d6954e1e60 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.768475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.769208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.769591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.769971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.782668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:05.794847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:05.795837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 92/177] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:05.796631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:05.869463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0799a85-61f6-4f27-81a2-a1d6954e1e60 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.872668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.910496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.911249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.911536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.911812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.912485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:05.912794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777bac35-922f-4fdc-8094-5fd0e0f15e2e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.919563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c130825-c502-4ced-aaf3-ce745f3aefa5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 86/178] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972310-3c4e-460e-bdb7-531f9c565284 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972310-3c4e-460e-bdb7-531f9c565284 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55972310-3c4e-460e-bdb7-531f9c565284 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 87/179] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:05.927277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.927723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.928105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.944138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:05.946733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:05.948189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:05.963770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:05.964396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:05.964677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:05.964950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:05.966076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:05.967563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:05.969187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 93/180] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:05.969951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.017248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 88/181] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:06.019791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.078464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90119fb1-ffb9-4178-978b-92cba36b51e1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.116061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.116668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.116952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.117239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.117902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:06.118213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.119634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5c3dc5-914b-4fc7-adef-a2e99a9117cb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.121861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.129692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52707d-a156-427a-b9da-d5e7677bb22d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.129863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 94/182] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.130967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.131251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.131525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.150344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:06.151580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:06.176854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.177661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.178248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.178544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] PATH_INFO: `/v3/users/26e85b5ffec84d34acc86041db6d5307` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.179732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] RBAC: Authorizing `identity:get_user(user_id=26e85b5ffec84d34acc86041db6d5307)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.181184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.182960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 89/183] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/users/26e85b5ffec84d34acc86041db6d5307 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.183641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.199337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.226650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 95/184] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:06.227545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.298529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e0bad-6b55-4284-a84b-3534ec4c47f9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.333117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.333875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.334450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.334764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.335493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=8727d28e1a63419baaa4c9f20a7bc777) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:06.335805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.337256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.340053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b12162-c85d-449c-a7c3-44dcbf314f8d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.343209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 90/185] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/role_assignments?user.id=8727d28e1a63419baaa4c9f20a7bc777&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.343987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.384342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.384995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.385276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.385548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.391383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.401683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:06.410212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:06.411255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:06.412060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:06.417133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 96/186] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:06.418133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.470708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.508878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf348-d75f-40cb-9c56-870dfd37b766 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.551321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.551774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.552063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.552318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.562949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99ddfa-c56c-4c01-aea8-9cf8019618a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.569799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.571360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.573032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 91/187] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.573791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.606469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.607111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.607424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.607848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.663583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.679562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ed103-510d-48f8-90b0-4a77717dfc50 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.680723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.683215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.714715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.715343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.715621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.715890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.716987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.718424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.720014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 92/188] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.722770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.726214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.795992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 97/189] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:06.801186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.827403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906735c-f3bb-46ca-9ea4-5b0d80ed13a3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.867328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.867328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.867515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.867658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.868343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:06.868804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:06.870183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:06.878813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 93/190] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:06.879589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:06.899437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.914645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569f2e19-d227-4a6f-a397-1f3f0af3d754 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:06.948481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:06.949075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:06.949355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:06.949624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:06.952921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:06.992281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dad11-2fff-411b-b9b6-9dc707399617 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.015272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.016732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.029135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.030347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.030795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.031324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.032179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.034045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.056982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 94/191] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:06 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:07.058068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.141323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 98/192] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:07.147408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.159242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.187977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.197765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716ccc3-2d71-4423-8e7b-fa6f60e4d0e6 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.238232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.239211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.239630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.240042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] PATH_INFO: `/v3/projects/cb2fe3eaa1ad46018859e190dc89800a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.241457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] RBAC: Authorizing `identity:get_project(project_id=cb2fe3eaa1ad46018859e190dc89800a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.243574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.245394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 95/193] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/projects/cb2fe3eaa1ad46018859e190dc89800a => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:07.246444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.268796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3d538-df54-4002-b083-4555f9c309e3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.310518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.311305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.311583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.313093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.313653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.317214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.317214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 99/194] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:07.317604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.401317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38277f-6b10-416f-97d4-786d516be1fb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.445313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.446376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.446883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.447328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.448752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.450453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.451796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303fb74-d8f7-4661-bec7-a48782981200 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.452510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 96/195] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:07.453574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.458682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.470937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.499859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.500773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.501059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.501335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.502128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=cb2fe3eaa1ad46018859e190dc89800a, user.id=26e85b5ffec84d34acc86041db6d5307) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:07.503117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.505463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.514422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 100/196] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/role_assignments?user.id=26e85b5ffec84d34acc86041db6d5307&scope.project.id=cb2fe3eaa1ad46018859e190dc89800a&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:07.516838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:07.653062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3981ad-82c8-42bd-9001-43ed99202bef None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.681855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56abb499-25ff-42ba-9d5d-3d9891efd9bf None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:07.717275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.719835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.720318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.720798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.721852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=e6ff617de01a4cf98cf9036f4a63e5f8) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:07.724642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.728321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da7189-d8f7-48ae-bba3-f4e73147a59e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.740020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 101/197] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/role_assignments?user.id=e6ff617de01a4cf98cf9036f4a63e5f8&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:07.755496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:07.756470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.756953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.757406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.860851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:07.863260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:07.876807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.879742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee149fbe-1187-4254-9774-6131f7404622 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.880056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee149fbe-1187-4254-9774-6131f7404622 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.880331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee149fbe-1187-4254-9774-6131f7404622 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.881804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 102/198] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:07.890963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:07.891270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:07.891526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:07.924270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:07.925928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:07.962947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:07.989991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 97/199] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:07.995847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.039882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 103/200] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:08.077883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.168357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ef127-23a0-47ba-b310-6ace79118175 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.181625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:08.229300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f71de-c1bf-496b-b120-c69ab5bc9410 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.247854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.248556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.248849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.249222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.255263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:08.256726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:08.258374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 98/201] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:07 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:08.264144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.268507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.269434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.269906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.270380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.281450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:08.287128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:08.287987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:08.288789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:08.291015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 104/202] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:08.296725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.322876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:08.404226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606b08f3-8772-4b07-8130-1b42fa10f95a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.439848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.441338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.441814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.442324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.443213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:08.443725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:08.445813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:08.455070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 105/203] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:08.461392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.499390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b421a-d606-41e2-9158-9bd691b11c41 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.572482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.573098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.573381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.573658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.574455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=e6ff617de01a4cf98cf9036f4a63e5f8) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:08.574793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:08.576237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:08.585063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 99/204] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:08 2022] GET /identity/v3/role_assignments?user.id=e6ff617de01a4cf98cf9036f4a63e5f8&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:08.598976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:08.622406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3019f3-1f22-42d7-bdcd-d55902b58028 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.679196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.679852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.680135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.680410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] PATH_INFO: `/v3/users/098934999774498399ba28290b19d6cc` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.684281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] RBAC: Authorizing `identity:get_user(user_id=098934999774498399ba28290b19d6cc)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:08.685842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:08.687550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 106/205] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:08 2022] GET /identity/v3/users/098934999774498399ba28290b19d6cc => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:08.693428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:08.750939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:08.835575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efa06f-3cf6-4ff5-8fb4-96ae91cdcb67 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:08.883965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:08.884653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:08.884933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:08.885206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:08.963248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:08.964668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:09.027217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:09.089548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 100/206] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:09.095299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:09.224398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c957a-1cee-433c-9721-77631cae3f5d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:09.267188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:09.267914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:09.268099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:09.268376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:09.271383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:09.272849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be2be6b-7508-458c-88eb-643fecf156c0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:09.275115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 107/207] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:09 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:09.280580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:09.411288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d2618-af34-4106-8112-70cc8ad24647 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:09.460738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:09.461352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:09.461635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:09.461914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:09.462681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, scope.project.id=41aabf5e702c472589f3e04b3efb7e10, user.id=098934999774498399ba28290b19d6cc) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:09.462997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:09.464520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca474be-dadf-4a6f-98ff-1d92325cb045 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:09.475518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 101/208] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:09 2022] GET /identity/v3/role_assignments?user.id=098934999774498399ba28290b19d6cc&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:09.858048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d041-5f3f-4f32-887b-485d0a529807 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:09.858399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d041-5f3f-4f32-887b-485d0a529807 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:09.858609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91d041-5f3f-4f32-887b-485d0a529807 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:09.860140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 108/209] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:09.871579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:09.871866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:09.872145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:09.879504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:09.893008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:09.894264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:09.963209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 102/210] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:09.988597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91d041-5f3f-4f32-887b-485d0a529807 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.012901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.105798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91d041-5f3f-4f32-887b-485d0a529807 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.144961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.145522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.145803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.146076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.160917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-49fb5231-206b-48ea-b00c-2263b459a7aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:10.166499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:10.167413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:10.168198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 109/211] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:10.173969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.194560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3d2b5-0446-45bf-b7c1-ddd18592d503 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.194868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3d2b5-0446-45bf-b7c1-ddd18592d503 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.195150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3d2b5-0446-45bf-b7c1-ddd18592d503 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.196484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 110/212] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:10.201317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3cdf-fe27-411c-a890-fad45557c36e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.201598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3cdf-fe27-411c-a890-fad45557c36e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.201874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd3cdf-fe27-411c-a890-fad45557c36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.219934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d2fd3cdf-fe27-411c-a890-fad45557c36e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:10.221289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d2fd3cdf-fe27-411c-a890-fad45557c36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:10.285344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce915c72-fb1b-4210-b9b9-3370ba058d70 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.292661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 111/213] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:10.293745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde097-3ff8-4407-abef-598cd6bee3d3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.294034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde097-3ff8-4407-abef-598cd6bee3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.294308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bde097-3ff8-4407-abef-598cd6bee3d3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.298617 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 112/214] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:10.301269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.301553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.301837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.319317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:10.320600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:10.323690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.327214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.328085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.328426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.328708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.329446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:10.329753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.331172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:10.333169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.342288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 103/215] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:10.348093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.394621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 113/216] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:10.394621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.460133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddc1fe-6020-4797-8c1f-fb024b499668 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.494886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.495508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.495785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.496058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.497153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.498637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:10.500341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 104/217] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:10.505835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.506144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac70df-4fea-4a57-b4df-f3179591897a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.542496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.543152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.543433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.543706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] PATH_INFO: `/v3/users/11102c0e975e466abafd70f6922219ae` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.555602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.560958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] RBAC: Authorizing `identity:get_user(user_id=11102c0e975e466abafd70f6922219ae)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.562885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:10.564678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 114/218] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/users/11102c0e975e466abafd70f6922219ae => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:10.565249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.579778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.620969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c7367-1ac2-4410-8460-13bdca7c85c3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.657849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.658482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.658795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.659074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.666963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc43a85-3d3d-4389-95b3-dbb07e92ff1c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.670493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:10.675770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:10.676662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 105/219] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:10.679583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.707939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.708539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.708815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.709083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.709723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.711232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:10.734902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 115/220] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:10 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:10.735991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.800261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.804308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343e996-5f0d-470a-bf61-7e7410ae9b31 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.832281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:10.846538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.847684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.848229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.848513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.854393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0754d24-8af4-4b26-b050-fa5df07837dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:10.903252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:10.903921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.904170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.904442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.905119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:10.905425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.906988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88846-e419-424c-a52f-53d0e688e824 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:10.916150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 116/221] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:10.917374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52a6b-2bd3-41b6-b252-a0ab91b82a65 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:10.917661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52a6b-2bd3-41b6-b252-a0ab91b82a65 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:10.917937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d52a6b-2bd3-41b6-b252-a0ab91b82a65 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:10.923281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 117/222] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:10.924266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52a6b-2bd3-41b6-b252-a0ab91b82a65 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:10.934250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:10.936146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a15fc4-3169-425f-9f15-7b64ad27005f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.054941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 106/223] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:11.056464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.056748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.057020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.061595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d52a6b-2bd3-41b6-b252-a0ab91b82a65 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.076196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:11.077433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:11.080961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.100192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.108664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.108664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.108664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.108664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.109856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.111920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.115354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 118/224] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:10 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.116469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.163753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 107/225] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:11.164760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.259444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51568d6-d2c8-4d34-aa20-353d8be5855a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.289519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8fd4a-3555-4d20-a8cd-93c6137be97a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.301323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.302036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.302404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.302765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.303955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.305587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.307648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 119/226] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.308572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.333491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.334398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.334705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.334981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.336781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.338793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.341131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 108/227] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.342124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.365850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.379646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.425035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66541d-873b-4dcc-9a42-026c7f8cdc96 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.468920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.469874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.470357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.470992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.483478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:11.490233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:11.491162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:11.491943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:11.499070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e0f2b9-dd9f-4521-8a45-47baf98f7e7f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.499203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 120/228] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:11.499707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.533186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.533544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.533827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.534096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.534943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=11102c0e975e466abafd70f6922219ae) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:11.535306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.536729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.546835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 109/229] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/role_assignments?user.id=11102c0e975e466abafd70f6922219ae&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.547808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.625782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f533966-cfac-4994-b557-b4c8cc0b50c9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.642123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.674102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.675207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.676214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.678267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/users/e6ff617de01a4cf98cf9036f4a63e5f8/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.682516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, user_id=e6ff617de01a4cf98cf9036f4a63e5f8, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.685911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.714271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 121/230] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:11 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/users/e6ff617de01a4cf98cf9036f4a63e5f8/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:11.731020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:11.743527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c9967-2918-4b8a-92df-1c6ca24f7323 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.781722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.782343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.782648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.782927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.783596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:11.783897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.785470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.794004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 110/231] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.799639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:11.942475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256481d4-6306-4e5c-8d17-7445e2880c76 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:11.978937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:11.979584 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:11.979910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:11.980192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:11.981370 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:11.982869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:11.984865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 122/232] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:11.990158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:12.104938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a1c7d-f096-4707-8393-cffabccc8a65 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:12.139620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:12.140258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:12.140544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:12.140823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:12.141971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:12.143455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482009a9-0583-45d3-8b92-ce954550c29b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:12.145419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 111/233] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:11 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:12.151454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:12.207122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:12.273525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c4d3f-3aae-48fa-beb8-09fd0565c475 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:12.307821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:12.308547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:12.308735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:12.309007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/users/e6ff617de01a4cf98cf9036f4a63e5f8/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:12.311224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, user_id=e6ff617de01a4cf98cf9036f4a63e5f8, role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:12.312648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069bba5c-0774-4bd8-84a8-39fc03d09a0c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:12.328313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 123/234] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:12 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/users/e6ff617de01a4cf98cf9036f4a63e5f8/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:00:13.227848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5ddb-de8d-4cf1-be81-26364f4a4fb6 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.228847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5ddb-de8d-4cf1-be81-26364f4a4fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.229327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf5ddb-de8d-4cf1-be81-26364f4a4fb6 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.231132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 112/235] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:13.236784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f08cc5-2ec8-403f-a0b2-075e75b2c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.237321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f08cc5-2ec8-403f-a0b2-075e75b2c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.237818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f08cc5-2ec8-403f-a0b2-075e75b2c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.256050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-67f08cc5-2ec8-403f-a0b2-075e75b2c9d0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:13.257563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-67f08cc5-2ec8-403f-a0b2-075e75b2c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:13.346099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 124/236] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:13.380290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf5ddb-de8d-4cf1-be81-26364f4a4fb6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:13.394161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31d8e-6578-4ab9-b2a8-8cc7c46fa83f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.394811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31d8e-6578-4ab9-b2a8-8cc7c46fa83f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.395810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31d8e-6578-4ab9-b2a8-8cc7c46fa83f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.397694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 125/237] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:13.403252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.403881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.404352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.406654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:13.443280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:13.444455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:13.495775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:13.504835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf5ddb-de8d-4cf1-be81-26364f4a4fb6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:13.572509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:13.573140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.573420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.573693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.582438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 126/238] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:13.588401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:13.593166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:13.594048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:13.594873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 113/239] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:13.602511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:13.615542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:13.728181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:13.779658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1098ea4-4797-4401-983e-6e968f2a117a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:13.811457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dac3c-03e1-405d-83b9-600bc55d4668 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:13.837482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:13.838135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.838412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.838711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.841460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:13.842990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:13.846100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 114/240] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:13.853009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:13.880389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:13.881246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:13.881659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:13.882062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:13.882896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:13.883355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:13.884863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:13.893494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 127/241] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:13.900111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:13.946938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.012675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0977b1-5356-4488-a83c-c622a1ad6741 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.057280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c4fc4-9453-41c1-a134-bcb7102eaa8d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.058542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.059504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.060688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.061156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.061992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:14.063668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f97785a-3380-435b-8ce3-b4f57d7a8935 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:14.087403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.093550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 115/242] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:13 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:14.095067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500a14-ca50-4ede-a0db-8b07d657bb3d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.095574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500a14-ca50-4ede-a0db-8b07d657bb3d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.096049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb500a14-ca50-4ede-a0db-8b07d657bb3d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.097641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 116/243] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:14.108281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.108936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.109219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.109514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] PATH_INFO: `/v3/users/098e4b39b7a64727a7bd05ad2672632c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.118372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.118909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.119353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.141724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] RBAC: Authorizing `identity:get_user(user_id=098e4b39b7a64727a7bd05ad2672632c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:14.143331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:14.151644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 128/244] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:13 2022] GET /identity/v3/users/098e4b39b7a64727a7bd05ad2672632c => generated 297 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:14.152715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.162236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:14.163504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:14.282297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.302719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 117/245] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:14.308857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35720a-1b4f-426e-81a3-db9471e2c928 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.321731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.351390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.352243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.352526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.352830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.439743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:14.441203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e5df2-fba5-4dde-af6d-8ece7667b791 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:14.448079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db3296-21f7-4574-bc8c-1045a98e940a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.486460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.496664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.497602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.498072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.498508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.512824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-23858bcb-1de3-44c0-b5b0-eaad22af66de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:14.519732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:14.521407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:14.522379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:14.524915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 118/246] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:14.526813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319b00-1ffe-4ceb-b56e-aa2b56fd4e50 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.527264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319b00-1ffe-4ceb-b56e-aa2b56fd4e50 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.527670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f319b00-1ffe-4ceb-b56e-aa2b56fd4e50 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.529327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 119/247] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:14.533923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319b00-1ffe-4ceb-b56e-aa2b56fd4e50 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.549659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.576819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 129/248] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:14.578525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.579031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.579796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.602030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:14.603831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:14.669545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f319b00-1ffe-4ceb-b56e-aa2b56fd4e50 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.695748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 130/249] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:14.702595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.727169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.728150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.728608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.729054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.729902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:14.730424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:14.732098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:14.747411 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 120/250] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:14.748592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.807755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.846798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3f66-6fb0-42b3-82fd-348cd8668d5e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.874971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:14.880039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf498ac-b4e2-4651-88ef-7f48de8ee535 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:14.900011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.900011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.900725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.901071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.902311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:14.903864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:14.906495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 131/251] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:14.907635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:14.927896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef057-8125-4211-a1cc-91018783347d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:14.929119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef057-8125-4211-a1cc-91018783347d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:14.929574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef057-8125-4211-a1cc-91018783347d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:14.930047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ef057-8125-4211-a1cc-91018783347d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:14.941494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ef057-8125-4211-a1cc-91018783347d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-563ef057-8125-4211-a1cc-91018783347d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:14.947843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:14.948707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:14.949508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:14.951736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 121/252] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:14.956873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef057-8125-4211-a1cc-91018783347d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.021366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d47b1a-ca33-4fe6-9cf6-a5f5fda8f69f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.056348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.057342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.057818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.058322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] PATH_INFO: `/v3/users/11102c0e975e466abafd70f6922219ae` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.059752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] RBAC: Authorizing `identity:get_user(user_id=11102c0e975e466abafd70f6922219ae)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.061435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111aa58f-9475-4080-bdd3-705650a15c9e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.063682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 132/253] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/users/11102c0e975e466abafd70f6922219ae => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:15.065494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4523ce-944c-4e84-bc83-39c51491b732 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.065995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4523ce-944c-4e84-bc83-39c51491b732 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.066457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4523ce-944c-4e84-bc83-39c51491b732 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.068891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 133/254] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:15.069946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4523ce-944c-4e84-bc83-39c51491b732 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.079730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ef057-8125-4211-a1cc-91018783347d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.100324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:15.128526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.129456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.129562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:15.130830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.131262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.132457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=098e4b39b7a64727a7bd05ad2672632c) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:15.132904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.134963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.142132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 122/255] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:14 2022] GET /identity/v3/role_assignments?user.id=098e4b39b7a64727a7bd05ad2672632c&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:15.143498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.225312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4523ce-944c-4e84-bc83-39c51491b732 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.305184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.305812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.306091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.306360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.307066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:15.307368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.309110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282c9e23-9aaa-40de-8be1-6ac016d0615a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.322661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 134/256] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:15.324262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.324738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.325217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.340236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682e51f-cffc-4ec0-ac5d-442630a2e90b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.344572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:15.345980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:15.385077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.386276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.386276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.386276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.391403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.392942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.394939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 123/257] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:15 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:15.395531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.420157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 135/258] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:15.421593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.463560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:15.523091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:15.557222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4907f5-8781-417c-ac0c-c9858177e3aa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.631701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.632743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.633214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.633695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/users/11102c0e975e466abafd70f6922219ae/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.640067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, user_id=11102c0e975e466abafd70f6922219ae, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.641887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.650216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327ce90-e7e5-40e1-b818-eb94835b3914 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.666954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 136/259] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:15 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/users/11102c0e975e466abafd70f6922219ae/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:15.668150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.732743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:15.734359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:15.735705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:15.736173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:15.738645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:15.741282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:15.751147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 124/260] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:15 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:15.757353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:15.931929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:15.939286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9a08f-7df7-4802-b8e9-c09536b3793a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:15.970553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5e308-800d-485e-bf5f-867d6e4501b0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.000668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:16.009819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.010502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.010803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.011071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.035178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:16.041437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:16.042405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:16.043895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 137/261] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:16.048466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:16.060301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.061347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.061861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.062333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.196329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:16.198127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:16.282238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86685a8f-0c9f-4e50-88b3-7b4521f55c2e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.309700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 125/262] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:16.315669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:16.346676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:16.360911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.371290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.371804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.372279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.373177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:16.373681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:16.375284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:16.393463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 138/263] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:16.403547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:16.449705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:16.551075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01103dda-fcd0-44a5-b990-136b2622e5ef None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.595632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.596621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.597102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.597571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.599625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:16.601150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:16.603427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 126/264] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:16 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:16.609087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:16.611997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6687d9-13ca-417f-8f9c-2b794615bd77 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.648929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.649947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.650389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.651106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] PATH_INFO: `/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.652486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] RBAC: Authorizing `identity:get_user(user_id=e6ff617de01a4cf98cf9036f4a63e5f8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:16.654216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:16.656228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 139/265] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:16 2022] GET /identity/v3/users/e6ff617de01a4cf98cf9036f4a63e5f8 => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:16.663471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:16.757772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:16.793990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86077b-3ed7-4a12-9ffa-b199460b190f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.815833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:16.839666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2687-da84-4de5-a69f-9f914d97e32e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:16.867955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.868567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.868846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.869118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:16.869849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=e6ff617de01a4cf98cf9036f4a63e5f8) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:16.870159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:16.871618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:16.877566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 127/266] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:16 2022] GET /identity/v3/role_assignments?user.id=e6ff617de01a4cf98cf9036f4a63e5f8&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:16.916244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:16.916866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:16.917152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:16.917429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.013550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.015041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94061237-01ed-4ceb-9657-c05030a87913 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.117627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.132023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 140/267] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:17.137660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.206673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08228-578f-4913-92a1-01afc1307bc4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.207018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08228-578f-4913-92a1-01afc1307bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.207241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b08228-578f-4913-92a1-01afc1307bc4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.208869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 141/268] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:17.214889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2271-ccde-4c7d-a60a-f83a2eb1f285 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.215174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2271-ccde-4c7d-a60a-f83a2eb1f285 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.215455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb2271-ccde-4c7d-a60a-f83a2eb1f285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.255674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b7fb2271-ccde-4c7d-a60a-f83a2eb1f285 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:17.256886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b7fb2271-ccde-4c7d-a60a-f83a2eb1f285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:17.277489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2d3cb-ff2b-4ef3-9b96-a0976e2b190e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.314434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.315096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.315511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.315786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.316930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.319061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.320507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 128/269] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:17.331541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.349009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.353434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 142/270] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:17.359185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0acd62-edbf-40a7-ad90-74762e65f2db None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.359480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0acd62-edbf-40a7-ad90-74762e65f2db None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.359754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0acd62-edbf-40a7-ad90-74762e65f2db None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.361278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 143/271] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:17.371774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.372055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.372331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.403547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:17.404715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:17.443669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db296f54-2665-4fa7-8c17-723d6b85a999 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.474077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 144/272] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:17.475478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.488123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.488732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.489111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.489441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.490346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=e6ff617de01a4cf98cf9036f4a63e5f8) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:17.490676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.492112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.506719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 129/273] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/role_assignments?user.id=e6ff617de01a4cf98cf9036f4a63e5f8&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:17.526630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.539219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.590069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dd808-4690-4ca6-aaba-98ec0de3f909 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.665304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.666326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.667050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.668915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.690665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:17.696880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:17.697808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:17.698619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:17.707081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 145/274] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:17.713748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.715707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.746778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816614-9ffe-447f-b9b1-f04f78e50208 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.787549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.788478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.788832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.789104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.790253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.791670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbcebc-0bc5-4a7f-b8e8-831e28efc2b9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.793255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 130/275] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:17.794461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b64b-6738-4e84-92dd-779f775b936b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.794768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b64b-6738-4e84-92dd-779f775b936b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.795038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881b64b-6738-4e84-92dd-779f775b936b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.796349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 131/276] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:17.799800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b64b-6738-4e84-92dd-779f775b936b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:17.848517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d591-3855-42b5-a791-298b72fe17e6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.861883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.890891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.891509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.891795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.892072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.892746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:17.893051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.894435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a02ab72-1de7-4d1b-9fb5-1e584e50c2aa None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.906749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 146/277] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:17.908772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.909347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.910022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.925953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881b64b-6738-4e84-92dd-779f775b936b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:17.927380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:17.929075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:17.962165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:17.962811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:17.963196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:17.963391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:17.963992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:17.965413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:17.985016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:17.990456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 132/278] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:17 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:00:17.991337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.010194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 147/279] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:18.032971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.116273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58997c-31bb-4bc9-be00-e58d8c8fafe8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.155846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.156612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.156891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.157164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] PATH_INFO: `/v3/users/5324c5c2d9304db79c9f6136baa05e6d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.169101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:18.172738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc397e-267d-46cf-a0f3-ced06dd4f853 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.174347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] RBAC: Authorizing `identity:get_user(user_id=5324c5c2d9304db79c9f6136baa05e6d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:18.175959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:18.178216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 133/280] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:17 2022] GET /identity/v3/users/5324c5c2d9304db79c9f6136baa05e6d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:18.185664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.211794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.212655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.213073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.213481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.232047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:18.236828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:18.237730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:18.238551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 148/281] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:18.244260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.258211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:18.321830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3fb1c5-a93f-4c1d-a609-710002bd4613 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.362616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.363241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.363484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.363756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.364483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, user.id=5324c5c2d9304db79c9f6136baa05e6d, scope.system=all) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:18.364790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:18.366256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:18.372577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 134/282] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/role_assignments?user.id=5324c5c2d9304db79c9f6136baa05e6d&scope.system=all&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:18.373955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ac138-6c97-40a1-930e-6bb97b3d1328 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.413483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.414103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.414379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.414681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.415361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:18.415659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:18.417054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa8c85-e63e-4395-a203-e6b580247671 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:18.426916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 149/283] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:18.433119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.548138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03e972-bc9c-4672-8c54-ba61421c3e7b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.569231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d93b2-fecf-48f1-8a5c-f13e3c8d1467 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.569730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d93b2-fecf-48f1-8a5c-f13e3c8d1467 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.569802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d93b2-fecf-48f1-8a5c-f13e3c8d1467 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.571376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 150/284] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:18.576493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.576777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.577051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.578507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:18.592353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.593329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.593827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.594304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] PATH_INFO: `/v3/users/098e4b39b7a64727a7bd05ad2672632c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.595695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] RBAC: Authorizing `identity:get_user(user_id=098e4b39b7a64727a7bd05ad2672632c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:18.596916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:18.597590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:18.598095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:18.600480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 135/285] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/users/098e4b39b7a64727a7bd05ad2672632c => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:18.606135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.617617 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:18.720432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 151/286] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 11:00:18.759920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e42339-ec22-4c18-9b7a-444dbbce13c8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.768895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.806498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.807159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.807433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.807702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.809126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:18.810853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:18.812792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 136/287] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:18.819494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:18.912117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03055c54-55e6-4394-988b-fae41ff56c2e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:18.955253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:18.955887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:18.956163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:18.957741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:18.957847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:18.976978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:18.981875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:18.982828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:18.983610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 152/288] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:18.991207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:19.021513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:19.043493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874cf76-98fe-4617-916f-2525cb332f0c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:19.120518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:19.121495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:19.121495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:19.121778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/users/098e4b39b7a64727a7bd05ad2672632c/roles/332f5677bf6242369ec55ecd88ac6ad7` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:19.123777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, user_id=098e4b39b7a64727a7bd05ad2672632c, role_id=332f5677bf6242369ec55ecd88ac6ad7)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:19.125226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:19.145351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 137/289] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:18 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/users/098e4b39b7a64727a7bd05ad2672632c/roles/332f5677bf6242369ec55ecd88ac6ad7 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:19.158271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f01cb-f4f2-4dbf-a38c-f69e4df39b77 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:19.241984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:19.242530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:19.242847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:19.243122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:19.243818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:19.244128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:19.245672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:19.259006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 153/290] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:19.264713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:19.399354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728dd00f-7289-42a1-87ad-ded486ef8130 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:19.417021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:19.439785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:19.440425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:19.440706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:19.440983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] PATH_INFO: `/v3/users/11102c0e975e466abafd70f6922219ae` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:19.442131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] RBAC: Authorizing `identity:get_user(user_id=11102c0e975e466abafd70f6922219ae)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:19.444232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:19.445968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 138/291] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:19 2022] GET /identity/v3/users/11102c0e975e466abafd70f6922219ae => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:19.451836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:19.567414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:19.593956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cc9b1-b756-44a0-8f0d-9a4a8140a5ae None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:19.630093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:19.631081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:19.632091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:19.632583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:19.703560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:19.705416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:19.794947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:19.812807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 154/292] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:19.818864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:19.952233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c977e-90e5-4aa1-9ed1-69f4bd05ae33 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:19.999239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:19.999876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.000161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.000435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.007263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:20.008717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7395ee-9dfb-40dd-9fe3-f5cd9df53a71 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:20.010525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 139/293] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:19 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:20.016177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:20.176813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b267ae-7f59-46d3-b48c-fa82b6f9dbf1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:20.234706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:20.235476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.235754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.236024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.236765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=11102c0e975e466abafd70f6922219ae) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:20.237073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:20.238503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6b891-669f-4768-baf0-b9f40a116067 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:20.251760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 155/294] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:20 2022] GET /identity/v3/role_assignments?user.id=11102c0e975e466abafd70f6922219ae&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:20.597609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993899e-5c83-4a8d-aaae-920dea9f4852 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.598144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993899e-5c83-4a8d-aaae-920dea9f4852 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.598355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993899e-5c83-4a8d-aaae-920dea9f4852 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.601604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 140/295] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:20.607789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ceb19-0931-4ed1-bed5-da415ca73528 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.608605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ceb19-0931-4ed1-bed5-da415ca73528 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.609097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ceb19-0931-4ed1-bed5-da415ca73528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.634458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-366ceb19-0931-4ed1-bed5-da415ca73528 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:20.636008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-366ceb19-0931-4ed1-bed5-da415ca73528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:20.672780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:20.720432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 156/296] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:20.730172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d97bf-1617-43cb-b5f0-28474f8feb2a None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.730806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d97bf-1617-43cb-b5f0-28474f8feb2a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.731271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d97bf-1617-43cb-b5f0-28474f8feb2a None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.733022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 141/297] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:20.738314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452111-5e6b-4320-b6a4-ca8af7b963bd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.738617 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452111-5e6b-4320-b6a4-ca8af7b963bd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.738893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d452111-5e6b-4320-b6a4-ca8af7b963bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.742089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d97bf-1617-43cb-b5f0-28474f8feb2a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:20.762323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9d452111-5e6b-4320-b6a4-ca8af7b963bd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:20.763814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9d452111-5e6b-4320-b6a4-ca8af7b963bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:20.805731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:20.847770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 157/298] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:20.849215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fca8e-07d4-45ff-a235-4a94498289ca None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.849500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fca8e-07d4-45ff-a235-4a94498289ca None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.849772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fca8e-07d4-45ff-a235-4a94498289ca None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.851142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 158/299] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:20.860251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d97bf-1617-43cb-b5f0-28474f8feb2a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:20.862916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.863371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.864590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.886992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:20.888193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:20.901811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:20.903015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:20.903293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:20.903564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:20.917886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:20.923668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:20.924559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:20.925342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:20.929614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 142/300] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:20.930733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:20.967071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 159/301] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:20.968215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:20.996120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.049284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f151032-8cce-446a-a642-27c2bd7ffb66 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.063545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.073468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e8a6e-f351-4b7c-958e-27f351eb093a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.085418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.086045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.086322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.086611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.096867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:21.101543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:21.106613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 143/302] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:21.107544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.116653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.117271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.117548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.117820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.118490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:21.118822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.120196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.128793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 160/303] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.129547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.217101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7ce27-6c70-4aa9-9d33-293fb5dd9a32 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.225954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.237954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2880d78a-0da4-4cb9-b89c-d315c8d322d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.256717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.257352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.257637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.257912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.259081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.260818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.262551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 144/304] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.268109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.280335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.281245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.281530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.281804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.282459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:21.282792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.284140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.293156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 161/305] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.293760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.320109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.383370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fca96d-2810-4d20-b2a2-19b5a2c0546f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.401477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf1752-b4c2-43c6-bc7d-348b817b2cbb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.419450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.420077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.420352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.420675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] PATH_INFO: `/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.434036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] RBAC: Authorizing `identity:get_user(user_id=47e8ce4b24cc4ec396cb2d01ba78ddb6)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.435499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.438051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.438679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.438965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.439240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] PATH_INFO: `/v3/groups` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.439860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.441272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.441942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 145/306] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.442849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.466603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 162/307] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 10 11:00:21 2022] POST /identity/v3/groups => generated 229 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:21.466603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.483191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.572382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.575692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7431b6c-baa8-4de5-bb12-aac1552b5903 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.620783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.621593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.621971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.622227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afbeb0c-bd35-422c-a45a-88947d0f62ed None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.622646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] PATH_INFO: `/v3/users/5324c5c2d9304db79c9f6136baa05e6d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.623768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] RBAC: Authorizing `identity:get_user(user_id=5324c5c2d9304db79c9f6136baa05e6d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.626845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.630624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 146/308] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/users/5324c5c2d9304db79c9f6136baa05e6d => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.636022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.678287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.679755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.680046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.680479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.681066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, user.id=47e8ce4b24cc4ec396cb2d01ba78ddb6, scope.system=all) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:21.681359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.683495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305039b5-abf9-4474-9f2f-0d639bafda32 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.691649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 163/309] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/role_assignments?user.id=47e8ce4b24cc4ec396cb2d01ba78ddb6&scope.system=all&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:21.712534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0004b-ebea-46b3-b880-f8310260fb5d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.712814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0004b-ebea-46b3-b880-f8310260fb5d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.713093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0004b-ebea-46b3-b880-f8310260fb5d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.714855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 164/310] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:21.721304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.721589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.721864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.748719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:21.750607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:21.770792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b59b7b-b3fb-44d9-9f34-c34c6582cd81 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:21.818670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:21.819773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:21.820126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:21.820471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] PATH_INFO: `/v3/system/users/5324c5c2d9304db79c9f6136baa05e6d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:21.822219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5324c5c2d9304db79c9f6136baa05e6d, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:21.824076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:21.836991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 147/311] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 10 11:00:21 2022] PUT /identity/v3/system/users/5324c5c2d9304db79c9f6136baa05e6d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:21.850069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 165/312] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:21.877346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:21.923458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:21.991871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2dcd-b974-4f98-b693-8b59837985af None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.030140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.030805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.031086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.031360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.041703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:22.046364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:22.047313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:22.048097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 148/313] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:22.053442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:22.062500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:22.160394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bb1aa-e4d3-41c5-a4b9-f2332ad3308b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.196285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.196893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.197170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.197439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.198110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:22.198407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:22.199814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:22.213585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 166/314] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:22.214933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:22.308571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:22.318433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe94c83-861d-424a-8969-e34d491dc1a2 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.353999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.354663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.354942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.355210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] PATH_INFO: `/v3/users/098e4b39b7a64727a7bd05ad2672632c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.356341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] RBAC: Authorizing `identity:get_user(user_id=098e4b39b7a64727a7bd05ad2672632c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:22.357774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:22.362674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 149/315] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:22 2022] GET /identity/v3/users/098e4b39b7a64727a7bd05ad2672632c => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:22.365187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:22.431714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:22.467389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028e542-5eda-4826-85cb-b76ea291d0f2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.500357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.500958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.501237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.501508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.570661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:22.572098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:22.638936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:22.676182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 167/316] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:22.677695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:22.778608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649aedb4-5e52-44d4-b5d5-f12135cf9096 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.812122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.812744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.813019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.813290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.814405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:22.815857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5433d-bc4b-44dc-a4b5-898381690875 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:22.822678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 150/317] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:22 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:22.824191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:22.929515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb296d-0a2a-45f3-8cce-a3c2bb087011 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:22.963678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:22.964276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:22.964551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:22.964818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:22.965544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, scope.project.id=a820641ee995493ab7cfa927f2009fca, user.id=098e4b39b7a64727a7bd05ad2672632c) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:22.965850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:22.967290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1141eae-163e-45fc-80ae-af58b421002e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:22.973323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 168/318] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:22 2022] GET /identity/v3/role_assignments?user.id=098e4b39b7a64727a7bd05ad2672632c&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:23.589984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba473-6980-424c-b5e5-7c219fb279a3 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:23.590356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba473-6980-424c-b5e5-7c219fb279a3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:23.590532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fba473-6980-424c-b5e5-7c219fb279a3 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:23.592051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 151/319] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:23.596973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe009-30ba-4143-94dd-308a139db4c5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:23.597270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe009-30ba-4143-94dd-308a139db4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:23.597550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582fe009-30ba-4143-94dd-308a139db4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:23.615276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-582fe009-30ba-4143-94dd-308a139db4c5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:23.616455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-582fe009-30ba-4143-94dd-308a139db4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:23.642626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:23.685837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 169/320] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:23.707815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba473-6980-424c-b5e5-7c219fb279a3 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:23.737504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:23.810725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ff84-5b77-4f58-a735-4718df581a2b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:23.811044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ff84-5b77-4f58-a735-4718df581a2b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:23.811276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328ff84-5b77-4f58-a735-4718df581a2b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:23.813046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 170/321] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:23.813150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fba473-6980-424c-b5e5-7c219fb279a3 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:23.817583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:23.817864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:23.818135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:23.836046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:23.837316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:23.846895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:23.847470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:23.847752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:23.848022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:23.850617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:23.850617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:23.853669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 152/322] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:23.858904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:23.918614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 171/323] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:23.940709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:23.961521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:23.968052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bdb14-21bf-425d-8ac3-6d33b6f33c18 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.002266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.002895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.003175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.003445 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] PATH_INFO: `/v3/groups` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.004056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.005491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc3d2-76cb-4120-baab-b48ef32f10d1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.020635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 153/324] 10.222.0.49 () {60 vars in 1134 bytes} [Wed Aug 10 11:00:23 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:00:24.021812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143267d7-35c1-450c-be31-e4c0b0f26fbe None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.022092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143267d7-35c1-450c-be31-e4c0b0f26fbe None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.022366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143267d7-35c1-450c-be31-e4c0b0f26fbe None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.023611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 154/325] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:24.029452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.029739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.030016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.048896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:24.049489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:24.050863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:24.060717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e67767-1325-42ef-9347-df61b4448177 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.099873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.100949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.100949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.101183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.112586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:24.117533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:24.118485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:24.119436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 172/326] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:24.125497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.134408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 155/327] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:24.158829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.249403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3427485-74da-45e9-8f79-ccabe08e6d7d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.254607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:24.281672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc434b0a-7bcc-4cbc-8a4a-44c81e6908d3 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.294831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.295786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.296286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.296624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.297552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:24.297659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.299100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.309089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 173/328] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:24.315127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.324492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.325735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.326163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.326672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.331678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:24.340839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:24.347362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:24.348266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:24.349055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:24.352154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 156/329] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:24.357666 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.444580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169b092c-c0fc-44e0-807f-abd42b0b7a57 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.487582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.488794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.489438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.489807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] PATH_INFO: `/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.491354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] RBAC: Authorizing `identity:get_user(user_id=47e8ce4b24cc4ec396cb2d01ba78ddb6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.494210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.495449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1cac0b-cc4e-48cf-b58f-ac49de1860a5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.496414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 174/330] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:24.501939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.541557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.542423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.542815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.543153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.543987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:24.544243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.545850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.555922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 157/331] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:24.568101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.579926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:24.614513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:24.643375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f32f1e-dd3c-4d14-b0d8-a5086900bbc9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.691198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.691981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.692354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.692719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] PATH_INFO: `/v3/system/users/47e8ce4b24cc4ec396cb2d01ba78ddb6/roles/332f5677bf6242369ec55ecd88ac6ad7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.694318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47e8ce4b24cc4ec396cb2d01ba78ddb6, role_id=332f5677bf6242369ec55ecd88ac6ad7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.696358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.698870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2cc2c-8440-4b30-b462-ac7bc00bd19a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.712565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 175/332] 10.222.0.49 () {58 vars in 1334 bytes} [Wed Aug 10 11:00:24 2022] PUT /identity/v3/system/users/47e8ce4b24cc4ec396cb2d01ba78ddb6/roles/332f5677bf6242369ec55ecd88ac6ad7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:00:24.767475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.768202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.768419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.768682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] PATH_INFO: `/v3/users/5324c5c2d9304db79c9f6136baa05e6d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.769953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] RBAC: Authorizing `identity:get_user(user_id=5324c5c2d9304db79c9f6136baa05e6d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.771425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4d49db-4548-4529-a822-07395ceeb718 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.773046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 158/333] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/users/5324c5c2d9304db79c9f6136baa05e6d => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:24.779767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:24.912964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb73c0-ff2d-46f5-a3fa-f939dd6dfeec None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:24.955607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:24.956222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:24.956502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:24.956769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:24.957504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] RBAC: Adding query filter params (role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, user.id=5324c5c2d9304db79c9f6136baa05e6d, scope.system=all) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:24.957810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:24.959264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ede8b-e5d9-4638-b8ca-6dff10619841 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:24.972057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 176/334] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 11:00:24 2022] GET /identity/v3/role_assignments?user.id=5324c5c2d9304db79c9f6136baa05e6d&scope.system=all&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:26.324176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3164c-9e33-46fd-80b5-a3242e8d77b5 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.324684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3164c-9e33-46fd-80b5-a3242e8d77b5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.324789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3164c-9e33-46fd-80b5-a3242e8d77b5 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.326402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 159/335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:26.333564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.333999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.334199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.342303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:26.359321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:26.360503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:26.436991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 177/336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:26.472956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3164c-9e33-46fd-80b5-a3242e8d77b5 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:26.559043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:26.587355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3164c-9e33-46fd-80b5-a3242e8d77b5 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:26.630257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:26.630931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.631212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.631486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.648343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fbce2acc-a964-42b4-9ca2-7146ef72668b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:26.654184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:26.655137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:26.655935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 160/337] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:26.662029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:26.667153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21feeb-9aeb-4085-8d5b-4f99b36c4e61 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.667440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21feeb-9aeb-4085-8d5b-4f99b36c4e61 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.667713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21feeb-9aeb-4085-8d5b-4f99b36c4e61 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.669346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 161/338] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:26.674432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.674751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.675034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.707672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:26.708889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:26.815790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f9358-b66b-43eb-9a7a-3d4e0660f78d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:26.829074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 162/339] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:26.857427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:26.858378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:26.858839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:26.859110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:26.859823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:26.860123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:26.861497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:26.868691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:26.877366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 178/340] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:26.883702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:26.898425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:27.001200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:27.057119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679da899-f2e0-4753-8083-36a867ac5884 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.073095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8a005-5d61-4347-9cd3-14243c08f43d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.106963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.107650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.107891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.108181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.137623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.138272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.138551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.138848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.161998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:27.167988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:27.169630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:27.171207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:27.178138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 179/341] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:27.178138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:27.245213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:27.246845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:27.278621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:27.348920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:27.370008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62824-7993-44bb-9e10-8eb5652bd27e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.429815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 163/342] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:27.431372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:27.472607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.473225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.473506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.473781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.474639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:27.474949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:27.476448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:27.499737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 180/343] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:27.506992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:27.629208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a49e8-be4e-47b8-b405-11c2afa2029c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.702719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.703359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.703638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.703912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.705135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:27.706619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:27.708264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 164/344] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:27 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:27.715284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:27.729304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c507bb-3be7-469b-ab79-b0f864aaccd8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.762428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:27.774082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:27.786599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.786599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.786599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.786599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.911577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f05f-2f48-4138-a8de-8991e5959039 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:27.912482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:27.914340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5723360-eee1-49ed-aee2-93c38f12c2bb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:27.974198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:27.974864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:27.975147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:27.975425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:27.984803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:27.986342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:27.988254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 165/345] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:27 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:27.994653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.071904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 181/346] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:28.073174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17969e-7574-43fe-90a2-f2c9583493d3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.073456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17969e-7574-43fe-90a2-f2c9583493d3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.073731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17969e-7574-43fe-90a2-f2c9583493d3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.075001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 182/347] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:28.083474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17969e-7574-43fe-90a2-f2c9583493d3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.114731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:28.189746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:28.195102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5d5c06-3a79-4903-a35b-ef1b2d0ade61 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:28.240564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:28.241196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.241482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.241761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.242520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=41aabf5e702c472589f3e04b3efb7e10) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:28.243520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:28.245153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab76ad0-7b7d-43e2-a180-43836ef7cef9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:28.254611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 166/348] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:27 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:28.256273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17969e-7574-43fe-90a2-f2c9583493d3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:28.256510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134415-87fc-425c-9ea8-2c119b14982a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.261142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134415-87fc-425c-9ea8-2c119b14982a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.261142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b134415-87fc-425c-9ea8-2c119b14982a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.278064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0b134415-87fc-425c-9ea8-2c119b14982a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:28.279726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0b134415-87fc-425c-9ea8-2c119b14982a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:28.320622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:28.321320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.322113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.322392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.323869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:28.325333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ce6d12-cdeb-41b3-a123-dbd25f3f7422 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:28.329205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 183/349] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:28.330449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc7636-1136-4b1f-bf74-61fd3dec3aef None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.330816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc7636-1136-4b1f-bf74-61fd3dec3aef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.331024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc7636-1136-4b1f-bf74-61fd3dec3aef None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.332268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 184/350] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:28.343651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc7636-1136-4b1f-bf74-61fd3dec3aef None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.409931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 167/351] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:28.411248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.411534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.411810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.459002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:28.460217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:28.510958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:28.528191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc7636-1136-4b1f-bf74-61fd3dec3aef None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:28.556620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 168/352] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:28.557446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.593551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:28.594192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.594470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.594787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.596010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:28.597459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a157b1f7-23b8-4d92-ac38-91aa853355cf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:28.606628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 185/353] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:28.606628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1684f-5352-48d4-8765-d0bd5134a849 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.606628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1684f-5352-48d4-8765-d0bd5134a849 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.606628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1684f-5352-48d4-8765-d0bd5134a849 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.606628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 186/354] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:28.607011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1684f-5352-48d4-8765-d0bd5134a849 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.610135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:28.703775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bfe2ad-f58d-4665-8ca4-de9bdb7aa015 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:28.741994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1684f-5352-48d4-8765-d0bd5134a849 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:28.747720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:28.748440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.748788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.749128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.762603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-74459639-8d58-433d-8273-e11f46810ea0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:28.767868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:28.768801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:28.769615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:28.770254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 169/355] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:28.772837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.772837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.772837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.795842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:28.796191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:28.796986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:28.797382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:28.797569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:28.797974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:28.798895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=41aabf5e702c472589f3e04b3efb7e10) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:28.799342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:28.800967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:28.807766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 187/356] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:28.815154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.856799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:28.881332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 170/357] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:28.882452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:28.901591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.032036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ed8d4-90d0-495a-90ba-0bcb1263ed14 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.093745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b97b5-39ac-46db-aaf2-449638c5a392 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.099087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.099719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.099996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.100262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.121836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:29.134969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:29.135930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:29.136714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 188/358] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:29.137387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.167073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.167694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.167973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.168242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.170056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:29.170369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.171778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:29.187418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 171/359] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:29.188433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.274208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.275111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.313770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99673ff-c9ff-4210-a6df-148f20b7a4fd None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.318753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46042d15-a77d-4dce-a746-0098978777ba None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.363121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.363736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.364019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.364291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.365118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:29.365475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.365864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.366832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.366935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:29.367441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.367922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.378495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 172/360] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:29.379371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.382072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c1ad530b-a337-429c-be45-d9fc344734a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:29.388721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:29.389746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:29.390550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:29.395488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 189/361] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:29.395488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ef76-381d-42f5-bd25-abbb0211a3cc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.395488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ef76-381d-42f5-bd25-abbb0211a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.395488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176ef76-381d-42f5-bd25-abbb0211a3cc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.400535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 190/362] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:29.403001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ef76-381d-42f5-bd25-abbb0211a3cc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.521885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22c360-c7fe-4092-a026-fb0a5f760c1c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.534675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176ef76-381d-42f5-bd25-abbb0211a3cc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.569607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.570147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.570428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.570735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.577832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.579099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.579983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.580457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.581151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] PATH_INFO: `/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.582676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] RBAC: Authorizing `identity:get_user(user_id=47e8ce4b24cc4ec396cb2d01ba78ddb6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.584305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:29.588081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 191/363] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/users/47e8ce4b24cc4ec396cb2d01ba78ddb6 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:29.589111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.598857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.673057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.674927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:29.717620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4346fe15-9d81-443b-9f71-e949f6d1768f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.760296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.760922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.761202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.761473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.762150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:29.762450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.763949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb793a3-b789-431e-8a69-74e0f5a06933 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:29.780588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 192/364] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:29.781829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.782113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.783243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.814213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:29.815893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:29.833613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 173/365] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:29.835385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.882060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.907031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:29.915718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 193/366] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:29.916891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:29.956169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f3a7b9-9e55-46c2-b608-4006599e4ec6 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:29.995187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:29.995951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:29.996438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:29.996813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:29.997791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] RBAC: Adding query filter params (role.id=332f5677bf6242369ec55ecd88ac6ad7, user.id=47e8ce4b24cc4ec396cb2d01ba78ddb6, scope.system=all) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:29.998205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:29.999782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.017680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 174/367] 10.222.0.49 () {56 vars in 1308 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/role_assignments?user.id=47e8ce4b24cc4ec396cb2d01ba78ddb6&scope.system=all&role.id=332f5677bf6242369ec55ecd88ac6ad7 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.017680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.062869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bacc5-772c-4b95-b794-97db59bb1eec None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.101783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.103210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.103639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.104053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.133929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.144162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2dacc1-14af-4a4b-aee3-0496274ad8e3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.184872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.188921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.190963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.202729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.203586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.203935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.204277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.208417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.210143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.216783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 175/368] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.216783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.315549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 194/369] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:30.316997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.345785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8603e2-a476-4ac3-9e35-0f3d2b17de0d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.387013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.387610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.387895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.388174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.399635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:30.405812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:30.406737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:30.407547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:30.412850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 176/370] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:30.412850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.436519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.443335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.448206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a101951-a3bf-44b4-8bcb-75a6f5389313 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.487422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.488972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.489768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.490257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.492394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.495404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.497170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 195/371] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.497860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.539059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2ac3-54a5-498a-8dd4-c77d72d9f80f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.583813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.584457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.584744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.585026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.586179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.588182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.590509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 177/372] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.591492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.617304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95006-81a0-463d-989a-1543c41676a9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.659867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.660851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.661324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.661837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.662713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:30.663228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.664794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.674536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 196/373] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.675658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.703810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.715110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:30.715907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5514e35d-0474-4e13-ab3c-080b3a5bf489 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.757665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.758605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.758906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.759731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.760505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=a820641ee995493ab7cfa927f2009fca) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:30.760820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.762930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.774086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 178/374] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.777109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.851445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d6208-3638-4a13-b56d-6fffddb6dc7a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.902435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.903139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.903422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.903697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.906655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:30.908301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e2221-f228-47d1-9a18-ada72fcd9a78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:30.910363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 197/375] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:30.911537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e23a6-8466-47f2-9a54-92f1490c1a07 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.911819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e23a6-8466-47f2-9a54-92f1490c1a07 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.912092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e23a6-8466-47f2-9a54-92f1490c1a07 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:30.918750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 198/376] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:30.924720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e23a6-8466-47f2-9a54-92f1490c1a07 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:30.950205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b4fda-b762-460f-bc8f-91352f182e52 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:30.992749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:30.993771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:30.994277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:30.994814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.033919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.048824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.057937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e23a6-8466-47f2-9a54-92f1490c1a07 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.095185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.096710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.099510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.100124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.100405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.100679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.101418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=a820641ee995493ab7cfa927f2009fca) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:31.101900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.104301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b5406b-9df8-40d5-8c53-e383bf909b49 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.110726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 199/377] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:31.114695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.123112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.123388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.167840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:31.169305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:31.252098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 179/378] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:31.262370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.288299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 200/379] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:31.313621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.358760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.414486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4441-9e6f-403a-b807-fd2919c441a9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.448685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.457957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.458924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.459405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.459891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] PATH_INFO: `/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.460018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645976f2-ae9f-44cc-ab8e-19e64dc0d151 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.461349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] RBAC: Authorizing `identity:get_project(project_id=dd03b7da5c5c4a22928c3c3d2bce3325)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.463728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.471017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 180/380] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:31.472330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.500063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.501257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.501682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.502335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.516067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1b703dc3-adcc-4bd9-bec3-f356e7154642 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:31.522156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:31.523062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:31.523861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:31.527071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 201/381] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:31.528569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ec57-81e7-43f8-8cdb-5b6305a5db0f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.529078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ec57-81e7-43f8-8cdb-5b6305a5db0f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.529544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05ec57-81e7-43f8-8cdb-5b6305a5db0f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.532004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 202/382] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:31.534500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ec57-81e7-43f8-8cdb-5b6305a5db0f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.608195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b3a35-6d68-4a31-9596-94038c2f8b32 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.643181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.659196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.660183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.660658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.661118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] PATH_INFO: `/v3/groups/1c8e9f1c7796479e882f6285b71de943` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.663416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05ec57-81e7-43f8-8cdb-5b6305a5db0f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.667092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] RBAC: Authorizing `identity:get_group(group_id=1c8e9f1c7796479e882f6285b71de943)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.668854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dfa4a-6bc4-4039-a158-ab20e0b287ed None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.671844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 181/383] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/groups/1c8e9f1c7796479e882f6285b71de943 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:31.673238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.673729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.674260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.703106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.703690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:31.704094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.704570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.704995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.705252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:31.705950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:31.706389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.707912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.720003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 203/384] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:31.721053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.740237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.796815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 182/385] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:31.798057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.846336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b5552-29a4-4004-8f19-406bbc3b5998 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.883065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.883779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.884064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.884371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.885218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] RBAC: Adding query filter params (group.id=1c8e9f1c7796479e882f6285b71de943, role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=dd03b7da5c5c4a22928c3c3d2bce3325) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:31.885530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.886974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.893587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 204/386] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/role_assignments?group.id=1c8e9f1c7796479e882f6285b71de943&scope.project.id=dd03b7da5c5c4a22928c3c3d2bce3325&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:31.894691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:31.917640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e74f6-220e-473c-b40a-70e38c89925e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:31.947327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.978191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:31.979477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:31.979759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:31.980037 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:31.983641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:31.985492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:31.986645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:31.988694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 183/387] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:32.003892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.060936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab7149-2af6-471c-805d-74014836b297 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.102734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.103588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.104081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.104567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.123432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:32.128204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ec187-5b39-42eb-a323-68c1d23ae08c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:32.129854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:32.130780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:32.131565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:32.140744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 205/388] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:32.146530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.168844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.169472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.169862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.170140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.171369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.172806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.175776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 184/389] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:32 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:32.180833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.246823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:32.296896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45158b0-8715-4492-84a1-373d44f46aa3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.320114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68601038-cf9c-4a6d-9c45-cf4c322e4757 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.332898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.333516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.333798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.334067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.334768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:32.335073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.336466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.338937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:32.346623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 206/390] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:32.355570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.391753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.392424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.392710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.392980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/groups/7055a7f03100465eb7988bc0ce41bac9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.396029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, group_id=7055a7f03100465eb7988bc0ce41bac9, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.397513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.420071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 185/391] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 11:00:32 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/groups/7055a7f03100465eb7988bc0ce41bac9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:32.495570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56d65a-74e6-4612-94ce-ea043f4cd259 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.534186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.534870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.535153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.535475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.536761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.538236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.540356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 207/392] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:32 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:32.546940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.661340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fd64c-43a4-4bf5-8a27-9c3875a846d2 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.675524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:32.699954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.700599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.700888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.701172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.702406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.703951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399f6ad0-8f6c-44af-a050-5a82a5b42820 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.706000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 186/393] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:32 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:32.713185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:32.783356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:32.819989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05426519-66c7-4d2a-a948-1488916747ef None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:32.858609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:32.858609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:32.858609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:32.858609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10/groups/7055a7f03100465eb7988bc0ce41bac9/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:32.860183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] RBAC: Authorizing `identity:create_grant(project_id=41aabf5e702c472589f3e04b3efb7e10, group_id=7055a7f03100465eb7988bc0ce41bac9, role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:32.863205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784653-470b-43ba-8f4b-2a4aeb6747cb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:32.885313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 208/394] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 11:00:32 2022] PUT /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10/groups/7055a7f03100465eb7988bc0ce41bac9/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:33.589178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c820-774d-4540-afdb-a89f393039bc None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:33.589611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c820-774d-4540-afdb-a89f393039bc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:33.590003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7069c820-774d-4540-afdb-a89f393039bc None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:33.591717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 187/395] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:33.597590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:33.597880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:33.598156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:33.616920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:33.618113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:33.635542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ddab-12fa-4f8d-b3a0-b51d85021c26 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:33.635857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ddab-12fa-4f8d-b3a0-b51d85021c26 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:33.636138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b559ddab-12fa-4f8d-b3a0-b51d85021c26 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:33.637661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 188/396] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:33.642830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:33.643116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:33.643396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:33.668045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:33.669348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:33.708638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 209/397] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:33.735543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:33.801406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 189/398] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:33.819421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:33.825025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:33.877694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba3d39-5e2e-482e-92ba-78c745b886b3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:33.930280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:33.931027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:33.931311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:33.931583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:33.943523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:33.948947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:33.949838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:33.951252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 210/399] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:33.959984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.047352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a251f-c901-4161-bb56-e60831141a23 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.072493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.073923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b79a859-bc98-47e9-8395-cc00cbd80051 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.098985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.112015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.113058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.113568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.114051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.118687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.118687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.118687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.118687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.119074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:34.119074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.119074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.129394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 211/400] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:34.135454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.138568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:34.143601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:34.144554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:34.145395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:34.152510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 190/401] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:34.158271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.282449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0734c2-5559-45bb-b9da-4c9ba679812d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.312543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fe270-43f3-4d00-9cc4-42573bf3bdaa None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.326257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.327201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.327716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.328211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.329081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:34.329589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.331216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.342697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 191/402] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:34.351555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.370960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.370960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.370960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.370960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.371356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.371356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.371356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 212/403] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:34.377106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.396376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.429070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.486265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c1ab4-84d4-448c-aa81-0dc3ea9942f9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.513226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316bcf9b-d948-41ff-bf1f-80b76eafd156 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.522181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.526595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.526595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.526595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.526595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.526595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.527848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 213/404] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:34.533939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.567537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.568450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.568732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.569042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.570830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.573068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f9f7-bbcd-4d62-8a9f-1a12e8b2684e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.576110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 192/405] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:34.577324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f851-236f-4521-9ef1-6a44c8d2024d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.577611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f851-236f-4521-9ef1-6a44c8d2024d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.577886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7f851-236f-4521-9ef1-6a44c8d2024d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.579112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 193/406] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:34.582795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7f851-236f-4521-9ef1-6a44c8d2024d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.653085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78733-82e4-4e01-9b94-85757ab0eb5a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.668804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.692186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.692850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.693128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.693401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/groups/7055a7f03100465eb7988bc0ce41bac9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.698013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, group_id=7055a7f03100465eb7988bc0ce41bac9, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.699509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa013b0-5d38-4dd9-af2c-c9d941477d55 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.716715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 214/407] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 11:00:34 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/groups/7055a7f03100465eb7988bc0ce41bac9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:00:34.718287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.719615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.723008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.729749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:34.740055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7f851-236f-4521-9ef1-6a44c8d2024d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:34.763804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:34.766435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:34.813329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:34.814269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:34.815517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:34.815952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:34.818092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:34.820768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:34.826003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 194/408] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:34.830724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:34.910072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 215/409] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:34.960893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.000092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c9642-a9e9-4238-8576-25938a8a5c4d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.040159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.041102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.041694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.042220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca/groups/7055a7f03100465eb7988bc0ce41bac9/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.044764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] RBAC: Authorizing `identity:create_grant(project_id=a820641ee995493ab7cfa927f2009fca, group_id=7055a7f03100465eb7988bc0ce41bac9, role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:35.046423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:35.068854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 195/410] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 11:00:34 2022] PUT /identity/v3/projects/a820641ee995493ab7cfa927f2009fca/groups/7055a7f03100465eb7988bc0ce41bac9/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:00:35.115011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385a2ad-4f82-435c-a554-60afb41ae1dd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.118960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:35.156086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.156993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.157315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.157603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.169883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8e086662-8c33-4f7b-9799-a26b0396bfea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:35.174871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:35.175848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:35.176891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 216/411] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:35.182494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.192924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329de13d-b157-44a3-b0b3-16bf192a5607 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.193242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329de13d-b157-44a3-b0b3-16bf192a5607 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.193492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329de13d-b157-44a3-b0b3-16bf192a5607 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.195314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 217/412] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:35.200952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.201223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.201493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.231931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:35.234235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:35.255660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:35.358556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89899134-9ffd-4da6-87e4-c4e8afe35920 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.371172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 218/413] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:35.393113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.427371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.428336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.428827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.429306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.430204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:35.430762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:35.432888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:35.446022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 196/414] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:35.451151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.500785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:35.510557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8b16e-d93d-41e4-99d0-4fe8f91cb43a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.559964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.560171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.560485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.560663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.585078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:35.595904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:35.596795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:35.598219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 219/415] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:35.603378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.661595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:35.684505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782d6cd-1ab6-4c15-8173-6defc63742b3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.723727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.724613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.724895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.725172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] PATH_INFO: `/v3/groups/1c8e9f1c7796479e882f6285b71de943` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.726294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] RBAC: Authorizing `identity:get_group(group_id=1c8e9f1c7796479e882f6285b71de943)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:35.728253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:35.731024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 197/416] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/groups/1c8e9f1c7796479e882f6285b71de943 => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:35.737060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.812722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e7d43-2fee-4ef2-9169-84bcac10cc90 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.880801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.881418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.881696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.881969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.882662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:35.882969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:35.885531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:35.900111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 220/417] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:35.900447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d687e6e-4d1f-4d3e-bfd6-88b1235036a1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:35.906052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:35.935437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:35.961876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:35.962516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.962823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.963139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] PATH_INFO: `/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.964314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=dd03b7da5c5c4a22928c3c3d2bce3325)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:35.966057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed986e-d982-43c1-ac88-6e973aba5bb9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:35.968030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 198/418] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325 => generated 344 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:35.972959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed45a1-d173-45f5-a671-e1d12913732d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:35.972959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed45a1-d173-45f5-a671-e1d12913732d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:35.972959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed45a1-d173-45f5-a671-e1d12913732d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:35.975050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 199/419] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:35.975843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed45a1-d173-45f5-a671-e1d12913732d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.010933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.064171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058adbe9-6d10-49d2-bc57-d0fa3272ae76 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.096039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed45a1-d173-45f5-a671-e1d12913732d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.106639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.107238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.107517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.107788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.146270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.147284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.147739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.148183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] PATH_INFO: `/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325/groups/1c8e9f1c7796479e882f6285b71de943/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.150927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] RBAC: Authorizing `identity:create_grant(project_id=dd03b7da5c5c4a22928c3c3d2bce3325, group_id=1c8e9f1c7796479e882f6285b71de943, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:36.153049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9808b-ba0f-4fe8-962c-f45d1d7e2b29 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:36.178854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 200/420] 10.222.0.49 () {58 vars in 1442 bytes} [Wed Aug 10 11:00:35 2022] PUT /identity/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325/groups/1c8e9f1c7796479e882f6285b71de943/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:36.178854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.178854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.178854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.192104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:36.194305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:36.218045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:36.219292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:36.236917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.354804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 201/421] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:36.376199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 221/422] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:36.381903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.401811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.404045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.514080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21f943-fc18-4331-8bac-3ea493465f03 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.527752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e509889-80db-4fa8-a5bc-c4b02b6ada94 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.551069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.551937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.552326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.552654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.552986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.554223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:36.555746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:36.557693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 202/423] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:36.563173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.575265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.576572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.577314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.577779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.593590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:36.598402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:36.599362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:36.600194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 222/424] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:36.607827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.665651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.686296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8fe02-5bf9-4deb-9194-eecc8b289d5f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.733590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.735115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.735594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.736078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.736621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58c3169-e9ef-443b-8423-446955740075 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.743377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:36.745368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:36.747332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 203/425] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:36.753630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.791040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:36.791574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:36.791891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:36.792163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:36.792838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:36.793144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:36.794530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:36.811417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 223/426] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:36.815382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:36.890906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:36.963444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea0b5d2-edfc-4329-835b-5494ab84c592 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:36.997490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a52399-5c01-4e63-818c-5e9f89285ea8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.032494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.033110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.033394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.033669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.034409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=41aabf5e702c472589f3e04b3efb7e10) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:37.034750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.036188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c303d74a-0f86-4618-9f5b-8ccbd0e215fb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.052064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 204/427] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 611 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:37.062776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.063413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.063704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.063984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.137054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.138532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.181387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.232053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ee2f3-5af5-42c0-8428-bff003314f3d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.232701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ee2f3-5af5-42c0-8428-bff003314f3d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.233193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ee2f3-5af5-42c0-8428-bff003314f3d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.234810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 205/428] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:37.244148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.244749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.246054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.246775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 224/429] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:37.255525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.279077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:37.281628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:37.305883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.373629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 206/430] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:37.387825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e56cff-201d-4dda-8364-1fc2df5a2c4a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.399676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.424285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.424917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.425197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.425469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] PATH_INFO: `/v3/projects/41aabf5e702c472589f3e04b3efb7e10` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.428332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] RBAC: Authorizing `identity:get_project(project_id=41aabf5e702c472589f3e04b3efb7e10)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.428332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.434440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 225/431] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/projects/41aabf5e702c472589f3e04b3efb7e10 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:37.435504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.447418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.538468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8850ac-811d-4c0e-b97d-aba7b53d3baf None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.549541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0361e2-b020-420a-a25d-decc515b3bf9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.580733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.581293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.581574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.581843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.598305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:37.600411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.601043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.601324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.601604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.603549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c04f9d71-3e1c-45b9-ba1f-cdfdd83f8c5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:37.603648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:37.604581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:00:37.605364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:37.605975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f66dec-fbf7-4949-89c0-2a125aa73cc7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.606050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 207/432] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:37.606657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2e23-77e0-4ec6-893a-a845d41417a5 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.606818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2e23-77e0-4ec6-893a-a845d41417a5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.607186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2e23-77e0-4ec6-893a-a845d41417a5 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.614488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 208/433] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:37.614488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b2e23-77e0-4ec6-893a-a845d41417a5 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.614751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 226/434] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:37.617602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.618157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.618687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.621472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.643731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:37.645477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:37.744780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b2e23-77e0-4ec6-893a-a845d41417a5 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.748157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 227/435] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:37.749957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.755318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.790976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.791586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.791868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.792171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.792839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:37.793146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.794536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.803392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 209/436] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:37.804188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.874483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:37.878129 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed53bd2-4eae-4231-931a-446e3c65f80e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.912146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72507f4-8cad-41f8-bb81-62d0df6e9f14 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:37.913399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.913997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.914279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.914551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.915309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=41aabf5e702c472589f3e04b3efb7e10) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:37.915618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:37.917699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:37.923289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 228/437] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=41aabf5e702c472589f3e04b3efb7e10&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:37.924064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:37.951664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:37.952378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:37.952660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:37.952987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:37.971688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:37.980501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:37.981478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:37.982269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:37.985836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 210/438] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:37.996188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.018039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.055714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c9b6b-1067-4b70-a710-765a79b897b9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.092705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.093320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.093600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.093874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.129161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f849715-6b5a-4a6f-ae65-48ae431fbd67 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.161297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.172045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.172717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.172998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.173270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.173987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:38.174297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.175709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.186632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 211/439] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:38.190359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.190478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.191856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.272368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.310202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91358115-4627-4d90-9af4-1b3b225fe2da None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.318317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 229/440] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:38.324230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.353704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.356119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.356749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.357295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.437312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.448508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.450210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.454493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85917bed-c150-4dc6-afb0-5c9e0ce18a78 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.491566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.492428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.492706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.492977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.494101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.496109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.497892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 230/441] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:38.503090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.537719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.560978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 212/442] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:38.566430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.611546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6dcc8-90bb-4cb2-be1c-b7a7a8d5598b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.644278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.644935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.645218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.645495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.646642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.648087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad187ae6-e6a1-40b6-aa42-52a1dc275281 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.649640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 231/443] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:38.650755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0137d9-f1b7-4358-adfe-e40ac059d238 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.651039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0137d9-f1b7-4358-adfe-e40ac059d238 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.651313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0137d9-f1b7-4358-adfe-e40ac059d238 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.653029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 232/444] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:38.655786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0137d9-f1b7-4358-adfe-e40ac059d238 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.658306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.668501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62739ff-a50d-4ded-ada0-21b681f6d7af None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.701530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.702190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.702527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.702847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] PATH_INFO: `/v3/projects/a820641ee995493ab7cfa927f2009fca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.703997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] RBAC: Authorizing `identity:get_project(project_id=a820641ee995493ab7cfa927f2009fca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.705435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299e0fe-bba4-444a-b1a0-567bb71ff1db None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.707091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 213/445] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/projects/a820641ee995493ab7cfa927f2009fca => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:38.708171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.708655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.708781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.726222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:38.727430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:38.754428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0137d9-f1b7-4358-adfe-e40ac059d238 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.756668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.789641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.790233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.790514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.790815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.791537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=5235ad50c96b41abb9b58fbdeb2ec7a9, scope.project.id=a820641ee995493ab7cfa927f2009fca) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:38.791843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.794648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 214/446] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:38.794648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.795151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.803686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 233/447] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:38.815992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.898191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:38.913615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb81be-c1c0-43d5-8fca-fb20d4fcfe1e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.940018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991193b-667c-424c-88f6-4bdf90403eeb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:38.952079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.952639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.952923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.953200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] PATH_INFO: `/v3/groups/7055a7f03100465eb7988bc0ce41bac9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.954325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] RBAC: Authorizing `identity:get_group(group_id=7055a7f03100465eb7988bc0ce41bac9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:38.955798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:38.957558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 215/448] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/groups/7055a7f03100465eb7988bc0ce41bac9 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:38.963891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:38.979210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:38.979880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:38.980168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:38.980439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:38.992037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:38.996690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:38.997633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:38.998441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 234/449] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:39.003776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:39.036574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:39.094566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a39240-27e2-4398-82e2-aaf3bcb05ff6 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.118806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43fbbc3-32f2-4149-9e3a-53c228ddf79b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.131354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.131968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.132252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.132528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.134521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] RBAC: Adding query filter params (group.id=7055a7f03100465eb7988bc0ce41bac9, role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=a820641ee995493ab7cfa927f2009fca) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:39.134521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.135118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.143040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 216/450] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:38 2022] GET /identity/v3/role_assignments?group.id=7055a7f03100465eb7988bc0ce41bac9&scope.project.id=a820641ee995493ab7cfa927f2009fca&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:39.163519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.164154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.164435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.164705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.165400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:39.165703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.167161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.177687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 235/451] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:39.183925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:39.215586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:39.297305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d7064-25d5-4c35-a784-20f21598a0ae None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.331052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.331663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.331939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.332208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.397176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.398679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.420904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:39.501839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 217/452] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:39.507329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:39.594130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:39.603770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b4ea7c-39eb-4d6f-a8c0-798afd60871a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.635157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.635761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.636038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.636852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] PATH_INFO: `/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.637949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] RBAC: Authorizing `identity:get_project(project_id=dd03b7da5c5c4a22928c3c3d2bce3325)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.639413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.640982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 236/453] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:00:39 2022] GET /identity/v3/projects/dd03b7da5c5c4a22928c3c3d2bce3325 => generated 344 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:39.646311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:39.739640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05ad18-2741-4d7b-b849-70f01d8e69be None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.759058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:39.772994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.773611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.773895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.774171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] PATH_INFO: `/v3/groups/1c8e9f1c7796479e882f6285b71de943` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.775294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] RBAC: Authorizing `identity:get_group(group_id=1c8e9f1c7796479e882f6285b71de943)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.776738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e49f77-01b6-42d8-82ae-4d22aa0ef533 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.778270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 218/454] 10.222.0.49 () {56 vars in 1180 bytes} [Wed Aug 10 11:00:39 2022] GET /identity/v3/groups/1c8e9f1c7796479e882f6285b71de943 => generated 229 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:39.784175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:39.879177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24e664-d321-4bcf-afe4-9cb4cc2b65d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:39.911905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:39.912495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:39.912778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:39.913049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:39.913773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] RBAC: Adding query filter params (group.id=1c8e9f1c7796479e882f6285b71de943, role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=dd03b7da5c5c4a22928c3c3d2bce3325) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:39.914113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:39.915558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79e91a-1b68-41cc-9e52-749034ec2fd8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:39.921100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 237/455] 10.222.0.49 () {56 vars in 1376 bytes} [Wed Aug 10 11:00:39 2022] GET /identity/v3/role_assignments?group.id=1c8e9f1c7796479e882f6285b71de943&scope.project.id=dd03b7da5c5c4a22928c3c3d2bce3325&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 611 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:42.392405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65a69c-fff6-49b4-98c8-04e38f768a03 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.392918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65a69c-fff6-49b4-98c8-04e38f768a03 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.392995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb65a69c-fff6-49b4-98c8-04e38f768a03 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.394542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 219/456] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:42.408206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96b9e-3f06-45bd-b457-deb584a77933 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.408642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96b9e-3f06-45bd-b457-deb584a77933 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.408817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96b9e-3f06-45bd-b457-deb584a77933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.430424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-23c96b9e-3f06-45bd-b457-deb584a77933 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:42.431696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-23c96b9e-3f06-45bd-b457-deb584a77933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:42.458224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:42.520676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 238/457] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:42.565470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65a69c-fff6-49b4-98c8-04e38f768a03 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:42.620975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c5f52-dd67-4191-b98d-b8238de34072 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.621834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c5f52-dd67-4191-b98d-b8238de34072 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.622070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c5f52-dd67-4191-b98d-b8238de34072 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.623591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 239/458] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:42.629348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac45c-e19c-43ae-bf82-b804495746b7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.629877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac45c-e19c-43ae-bf82-b804495746b7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.630345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac45c-e19c-43ae-bf82-b804495746b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.657934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d62ac45c-e19c-43ae-bf82-b804495746b7 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:42.659343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d62ac45c-e19c-43ae-bf82-b804495746b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:42.733214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb65a69c-fff6-49b4-98c8-04e38f768a03 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:42.788423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:42.788788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 240/459] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:42.789051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.789330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.789609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.789953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349adda-1c8b-403f-8f28-b28f43911807 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.790321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349adda-1c8b-403f-8f28-b28f43911807 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.790610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349adda-1c8b-403f-8f28-b28f43911807 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.796521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 241/460] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:42.797767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:42.798150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:42.798528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:42.816307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:42.817756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:42.831750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:42.832945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:42.834765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:42.835672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:42.836934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:42.836934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:42.836934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:42.836934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 220/461] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:42.837156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:42.889820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:42.925812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 242/462] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:42.934613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:42.942052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:42.976755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dc047-dcb6-44f0-b998-9ed17c83be5a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.020999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.021642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.021926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.022204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.038117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.039905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.047172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa1f83-233f-4237-9cca-d85ed9ecb301 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:43.058058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:43.059540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:43.060439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:43.060439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.060439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:43.064685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 221/463] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:43.065377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.088029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.088750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.088930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.089168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.089808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:43.090154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.091545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.108588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 243/464] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:43.109406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.168396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bf2d9-8d33-4f68-8bcc-517a60e5cabc None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.187956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.214475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.215674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.215972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.216225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.220498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da2bac8-5d16-489d-93b1-fb99708c5fc0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.227073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.228523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fa925ed2-41bf-45b4-95ee-71e70b3d505e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:43.243652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:43.244590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:43.245473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:43.245473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.245473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:43.245473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 222/465] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:43.246338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef33b-f0dd-46b8-b8df-df35e0158a71 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.246515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef33b-f0dd-46b8-b8df-df35e0158a71 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.246874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849ef33b-f0dd-46b8-b8df-df35e0158a71 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.248084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 223/466] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:43.253184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ef33b-f0dd-46b8-b8df-df35e0158a71 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.278742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.279349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.279779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.280052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.281007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:43.281568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.283657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.302813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 244/467] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:43.302813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.352648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849ef33b-f0dd-46b8-b8df-df35e0158a71 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.386501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.387158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.387434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.387706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.388340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.389771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96574f8-bcfc-4ac7-9ccf-7cd3f10260cc None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.407322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.416042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6967c053-0f2c-4247-bda0-05af8089c029 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.420186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 224/468] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:43.422047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.422534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.423058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.447776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:43.449113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:43.458053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.466961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.467939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.468438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.468887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.469941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:43.470506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.472286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.487877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 245/469] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:43.489158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.535056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 225/470] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:43.536201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.608336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d000db-3682-43bc-899d-4af0bb9963ac None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.650556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.651194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.651479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.651755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.652401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.653876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fae7a-5f58-47cc-bef5-48a305944a5e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.662938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79227f1d-645b-4051-83a1-3058649a5cf3 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.675984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 246/471] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:43.677515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98120-a4fa-4f81-8004-d211ffec19eb None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.677972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98120-a4fa-4f81-8004-d211ffec19eb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.678414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef98120-a4fa-4f81-8004-d211ffec19eb None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.679837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 247/472] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:43.680731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef98120-a4fa-4f81-8004-d211ffec19eb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.700115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.707958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.708732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.709126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.709471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.710196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.711903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c058ab2-6bdf-4dd7-b0ca-bb187830fe8e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.741125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 226/473] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/users => generated 248 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:43.742638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.742957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.743428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.754485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:43.766965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:43.768240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:43.866946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef98120-a4fa-4f81-8004-d211ffec19eb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:43.911235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:43.912249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:43.912744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:43.913019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:43.927200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:43.928679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:43.940794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 227/474] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:43.948010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:43.948978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:43.950503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:43.950503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:43.950503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:43.952499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 248/475] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:43.957794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:43.963419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:44.086389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc85add-1fdd-4463-8476-3b81618d3a23 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:44.096644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:44.100803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb25d118-80f9-48de-9441-3fcef776b512 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:44.107768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:44.130175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:44.131161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:44.131663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:44.132188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:44.133419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:44.133864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:44.135402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:44.149284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 228/476] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:44.151385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:44.152193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:44.153133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:44.153543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:44.154427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:44.155275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:44.156104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:44.170170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.sql.core [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:00:44.170170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 11:00:44.170170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [parameters: {'id': 'f4f3451deb094a26ab653bea5218971d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 11:00:44.170170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Aug 10 11:00:44.186374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:00:44.187511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application yield resource Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise self.value Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application session.commit() Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:00:44.188732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:00:44.189808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:00:44.195448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:00:44.198195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [parameters: {'id': 'f4f3451deb094a26ab653bea5218971d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:44.201442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 11:00:44.205487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.207831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 249/477] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 11:00:43 2022] POST /identity/v3/roles => generated 144 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:44.207831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:44.292564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4074e9-2514-4cdf-93c9-e420ea30a587 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:44.318840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57fdf3-9352-4b5d-9502-7cd15e1f8f90 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:44.342752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:44.347974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:44.348785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:44.348785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:44.349420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:44.350898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:44.376226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 229/478] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:44 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:00:44.388345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:44.389006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:44.389391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:44.389702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:44.412083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-efab1dbf-684f-456f-a5c3-bcc466b54540 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:44.417270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:44.418673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:00:44.419477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:44.422169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 250/479] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:00:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:44.426560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:44.485453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:44.581389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0481b9-7343-48f8-ab79-c76b4b294fd7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:44.616610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:44.617215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:44.617494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:44.617766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:44.618428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:44.618764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:44.620201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a405f-d774-4442-a098-a33481c4c466 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:44.628485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 230/480] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:45.797308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211442-40a2-4180-8ede-82f4173d7e7d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:45.797817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211442-40a2-4180-8ede-82f4173d7e7d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:45.797892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44211442-40a2-4180-8ede-82f4173d7e7d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:45.799446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 251/481] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:45.804719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:45.805101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:45.805262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:45.831491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:45.832661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:45.931549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 231/482] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:45.964687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44211442-40a2-4180-8ede-82f4173d7e7d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:45.998566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.100329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44211442-40a2-4180-8ede-82f4173d7e7d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.144615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:46.145237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.145867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.146137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.160382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5073184e-e262-473e-b65a-fd1055e7c703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:46.165355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:46.166248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:46.167088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 252/483] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:46.174737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.204944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.302319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3264ee01-e095-4b66-b115-509158f2ef70 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.307238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bfe67-3754-4f28-ad25-f29c36cfd7d5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.307796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bfe67-3754-4f28-ad25-f29c36cfd7d5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.308260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bfe67-3754-4f28-ad25-f29c36cfd7d5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.310556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 253/484] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:46.315887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.316436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.316923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.344774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:46.345984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:46.369230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:46.369752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.370298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.370989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:46.371295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:46.373184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:46.390361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 232/485] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:46.395939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.457444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 254/486] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:46.478630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.512076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.535010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.564706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f4cd2-aa05-419c-aff4-72aa2bf09e8c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.605522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:46.607507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.607802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.608119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.622700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:46.624523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-96d5d7cb-e3a8-4e3e-9e9b-1bc924d7d645 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:46.641716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:46.642647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:46.643481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:46.643481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:46.643481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:46.643651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f4bce-9aa6-4eaf-8ad1-61aec3ff33ee None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.643735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 233/487] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:46.647767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c9cd-8066-41c1-8ebc-c1882fb5aee8 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.648070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c9cd-8066-41c1-8ebc-c1882fb5aee8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.649097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c9cd-8066-41c1-8ebc-c1882fb5aee8 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.650342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 234/488] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:46.654993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c9cd-8066-41c1-8ebc-c1882fb5aee8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.692491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:46.694080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.694604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.695359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.709940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-31a4e3df-8c4c-42bb-bee9-042c2b7bcedd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:46.716978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:46.717860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:46.718694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:46.721624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 255/489] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:46.723651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.724223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.724739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.745901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:46.747445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:46.778067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c9cd-8066-41c1-8ebc-c1882fb5aee8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.816473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:46.817105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:46.817425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:46.817822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:46.818585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:46.819011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:46.820672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:46.829242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.835607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 256/490] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:46.836948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.841241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:46.842178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 235/491] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:46.847648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:46.965177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae00389-8725-400c-9329-a7320dc49af5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.970677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef5655-25c1-4e3b-b9f0-178b2383ca7c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:46.999898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.000512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.000788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.001065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.001730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:47.002034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.003432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.009576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.010380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.010828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.011251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.011368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 257/492] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:47.012168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.024789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.026437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-2ba64f31-803b-42d0-83ae-f72836c81220 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:47.039900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.041000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:47.042116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 236/493] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:00:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570599d-7583-406e-a9bd-6678a7da154f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570599d-7583-406e-a9bd-6678a7da154f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570599d-7583-406e-a9bd-6678a7da154f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 237/494] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:47.049167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570599d-7583-406e-a9bd-6678a7da154f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.093799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.114782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.129659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344d718-343b-4c8c-bf43-d87596f9e4e5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.162940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570599d-7583-406e-a9bd-6678a7da154f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.165798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.166428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.166753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.167024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.177250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:47.183154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.184069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:47.185390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 258/495] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.186057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.203854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.204478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.204758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.205030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.215554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.216993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-e9b9f7ab-5f80-48da-9495-0619c81e874a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:47.231302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:47.233952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:47.233952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:47.233952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:47.233952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 238/496] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.234278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.234565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.234857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.253790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:47.255011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:47.299391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37b897-750e-4597-92c8-55026c3bba7b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.325478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.326032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 239/497] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:47.326898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.343277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.343877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.344159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.344423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.345152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:47.345461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.346908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.368414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.383579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 259/498] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:47.384418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.426879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af2ea2-1fbc-45bd-aeb0-1f7ac1b65037 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.459783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.460381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.460659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.460927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.461584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:47.461883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.463267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466d5ed-1c08-4ad5-a0f0-1d7da9a3621e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.471098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 240/499] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:47.472152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531027f-1a74-4ced-8de0-f76a385adc4b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.472439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531027f-1a74-4ced-8de0-f76a385adc4b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.472711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531027f-1a74-4ced-8de0-f76a385adc4b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.473877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 241/500] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:47.474552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531027f-1a74-4ced-8de0-f76a385adc4b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.484975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586cde9-9caf-4848-8cbd-a7604cb786f4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.518389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.519031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.519311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.519581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.520219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:47.520520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.521883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.533138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 260/501] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:47.533905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.573750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531027f-1a74-4ced-8de0-f76a385adc4b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.575320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.579452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.608310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.609320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.609603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.609881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.619909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:47.625061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.625967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:47.626817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 242/502] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.627483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.638422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad97eb5-5736-4f7b-adf2-8bf978279357 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.671490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.672090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.672372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.672644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.724604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db23d71-5e0f-4362-9390-0b4812e80c49 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.737574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.739694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.757461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.758074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.758352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.758656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.768679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.770100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.782310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-67e5882e-6c83-42ad-857a-e995a7f30ae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:47.784901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.785952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:47.786845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:47.786845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:47.786845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:47.786845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 243/503] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.787217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.787425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.787695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.804966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:47.806141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:47.815531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:47.836464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 261/504] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:47.837371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.874649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 244/505] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:47.875485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:47.941744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6bd5dd-2ccc-4ed9-b14b-be56158509fc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.973457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeff43f-3ccb-4cdb-820d-aab4d4224593 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:47.975494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:47.976040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:47.976314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:47.976581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:47.983283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:47.984703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:47.997032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:47.998017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 262/506] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:47.998966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.002135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.012078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.012685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.013490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.013759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.014421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.015048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.016128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.023775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 245/507] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.024540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.033478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.098097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a181d1-7404-42f0-90e7-47fe2699f56a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.121786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baff7a11-822a-4c3f-9c54-651cf898e3ee None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.129759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.130356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.130664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.130936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.131571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.131877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.133227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.145006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 263/508] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.145790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.160001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.160632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.160916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.161193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.171634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.173069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:48.186561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:48.187480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:48.188335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:48.188335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.188335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:48.188335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 246/509] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:48.188547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.206597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.243724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa9fa1-433e-4256-a807-cf2b6776bedf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.244789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.277130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.277754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.278035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.278307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.286558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32322bb-161f-4fa4-9c2f-02a4463df7a6 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.288785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.290215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:48.304085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:48.304989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:48.306020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:48.306020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.306020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:48.306020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 264/510] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:48.306437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.321501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.322104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.322385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.322687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.323412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.323715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.325128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.360393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 247/511] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.361199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.404217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754c1ca-af2e-4012-b8a4-d706745048a5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.419776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.437768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.438387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.438707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.438986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.449131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.451145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.459170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02351e98-7603-49da-b0f5-8cf6d11a0b81 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:48.464581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:48.465454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:48.466314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:48.466314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:48.466314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:48.466314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 265/512] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:48.466535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.480216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.496691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.497748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.498025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.498295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.504998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.506430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:48.518193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:48.519279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 248/513] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:48.520160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.567978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fbaa6b-24a5-48c5-b932-8857e23e8adc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.600486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.601082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.601362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.601636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.602265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.602612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.603944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.615242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 266/514] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.616010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.617172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26648fda-f747-4c69-92d0-b06f1bf9daf7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.645027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.650708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.651309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.651591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.651868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.652507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.652811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.654242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.665982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 249/515] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.666789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.699755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.721287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b1a25-0545-4764-9381-6e3c8df6f2e0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.756121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.756714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.756995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.757270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.762689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae703f5-2d53-4ab3-a5ca-2c8301933618 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.796229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.796820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.797099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.797373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.798005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.798307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.799720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.811093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 250/516] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.811691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.826697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.828137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.853447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.911320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:48.918244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2e748-292e-4843-888f-46a98a9d67bf None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:48.930767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 267/517] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:48.931429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:48.954380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:48.955037 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:48.955316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:48.955586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:48.956311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:48.956617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:48.958046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:48.974421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 251/518] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:48.976173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.030699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb93c2-158f-4b7f-9eb0-4993a7d7df02 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.056221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.064512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.065130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.065414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.065691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.072246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.073677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.073774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3201de8-f7fc-4da7-b9b3-bd60b62160d8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:49.086255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:49.087326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 268/519] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:49.088285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.110544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.111194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.111470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.111741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.112371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.113797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.133858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 252/520] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:00:48 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:00:49.134669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.154883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.202103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920695a7-6f7a-452d-b703-15597f95e92d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.241451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.242098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.242382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.242683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.251352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.252854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.256226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d06d-218f-4a10-8a25-53d08aaf7635 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:49.265144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:49.266132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 269/521] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:49.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.300843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.301516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.301706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.301958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.315838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.317333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.323627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:49.338009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:49.338990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:49.339839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:49.339839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:49.339839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:49.340001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 253/522] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:49.341707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.388504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e89db-dc26-40bc-80f7-37b3547ba663 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.409134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.425707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.426323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.426623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.426900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.453142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897e466-76b3-456d-b22f-6cc601c60c1b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.487035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.487653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.487929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.488198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.488888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:49.489196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.490663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.498345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.499817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.508021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 254/523] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:49.508646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.554465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.613202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646246d8-f6f3-4bd9-92ff-c19a814c5d01 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.613397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 270/524] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:49.613397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.636563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.648676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.649362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.649714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.650060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.650890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:49.651269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.652776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.670672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 255/525] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:49.671865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.726848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1846bc-7152-4168-ad21-d689a72452c5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.768675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.769274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.769551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.769814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.770444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:49.770776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.772128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.776706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ce80a-ac44-4701-8f5f-e664149f738f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.786901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 271/526] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:49.788403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.800320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.814190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.814757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.815045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.815303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.816029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:49.816337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.817768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.823386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 256/527] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:49.827542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:49.907461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:49.913632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697eab7-c457-4f99-be36-98f271d2a722 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.952467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:49.953107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:49.953457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:49.953733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:49.965992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:49.968026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:49.985272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:49.986268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:49.987169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:49.987169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:49.987169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:49.989449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6682514-fa0c-48d7-aa01-0315768f2329 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:49.991007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 272/528] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:49.991901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.030626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.031346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.031633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.034601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.066020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.110349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.112068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.118211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7be86-8c2e-49e8-acd8-c512ee286fbf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.159377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.161326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.162678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.163034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.163377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.172369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.174670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:50.189433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:50.190496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:50.191585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:50.198596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 273/529] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:50.198596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.229197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 257/530] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:00:50.230496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.312584 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd009b-cdb2-412b-ae39-d437540bc8d5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.332817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.350139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.350779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.351063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.351311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b6a46-d7c9-47b3-8533-e966bd2ee1e9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.351489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.351980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:50.352284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.354740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.369585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 274/531] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:50.370626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.395188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.396088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.396372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.396647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.397333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:50.397639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.399369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.412647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 258/532] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:50.413567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.426599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.509824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbedb4-be37-4ef6-9fcb-f9988401b3c9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.544385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ac304-ba75-4bd6-8fcf-4ce5fd720547 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.554277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.554938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.555948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.556230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.569844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.571327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.585739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.586298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.586591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:50.586694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:50.587535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:50.588379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:50.588379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:50.588379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:50.588548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.588616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 275/533] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:50.588616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.599623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.601220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:50.613911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:50.614784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:50.615656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 259/534] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:50.616541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.621465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.687452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.693030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695798-7f6c-42f7-9683-d0a4761f7110 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.729869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57616c-ced0-4400-a2dd-26503e31ee33 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.735542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.736154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.736444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.736771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.737514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=eacd5dcb5de7474da98c2e7f30a4e6db) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:50.737827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.739276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.745401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 276/535] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/role_assignments?user.id=eacd5dcb5de7474da98c2e7f30a4e6db&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:50.746232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.788566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.789284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.789566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.789835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.790490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:50.790820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.792209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.816676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 260/536] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:50.822131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:50.860209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:50.867308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570253a4-8933-4c8c-90c1-ecaf2247db9a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:50.910229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:50.911335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:50.911624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:50.911966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:50.912697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:50.913005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:50.914452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:50.937764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 277/537] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:50.945661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.000856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b59c1a-ac73-4e41-84bf-9b939c7ef95b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.027209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:51.040368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.042611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.042611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.042611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.059671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:51.064681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:51.077977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:51.078919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:51.079844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:51.083852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 261/538] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:51.085105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.121301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982856f-020f-4302-9af3-92073deefbb7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.157060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.157664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.157937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.158205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.159193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=5882280742154859b52fa47c350a137f) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:51.159662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:51.161404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:51.167355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 278/539] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:50 2022] GET /identity/v3/role_assignments?user.id=5882280742154859b52fa47c350a137f&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:51.195746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c3b93-f6e3-467d-a445-36569f8beb24 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.234365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.235006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.235353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.235629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.236331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:51.236634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:51.238094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edaca7-4e25-4acf-bf49-231f6d816283 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:51.254712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 262/540] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:51.257557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.297029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:51.371978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad672d-4413-40bd-9878-edb66c4acd75 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.372277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad672d-4413-40bd-9878-edb66c4acd75 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.372564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad672d-4413-40bd-9878-edb66c4acd75 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.376140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 263/541] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:51.381257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.381756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.381990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.400117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca943b9-86e9-4496-816b-451962f135ce None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.401853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:51.403175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:51.437025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.437631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.438008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.438286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.439159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=5c5297e9a66d4124bc4888e5a92dc885) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:51.439465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:51.440900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:51.447197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 279/542] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:51 2022] GET /identity/v3/role_assignments?user.id=5c5297e9a66d4124bc4888e5a92dc885&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:51.454898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:51.517163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 264/543] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:51.555748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.665238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:51.684408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ee00fa-e626-427e-907f-62caf203aeb1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.724216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.724849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.725127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.725395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.741632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-62d293a8-7c86-4576-bf22-7c626b57fd2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:51.746861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:51.749042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:51.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 280/544] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:51.754771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.893168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b82c2b-7d1b-47f1-adb6-dc756bbc977c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:51.912105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedff3-ecc7-403a-8acd-5c702d83c068 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.912414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedff3-ecc7-403a-8acd-5c702d83c068 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.912653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fedff3-ecc7-403a-8acd-5c702d83c068 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.914238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 281/545] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:51.923999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.924554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.924554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.935202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:51.936452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:51.936736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:51.937006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:51.937729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:51.938029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:51.939423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:51.948396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 265/546] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:51.954906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:51.958172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:51.967634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:52.018955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.063574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc1e3b-3241-444c-821b-941bf29b53e5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.103825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.104712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.104992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.105263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.117775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:52.119603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:52.135212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:52.136137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:52.136972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:52.136972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:52.136972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:52.136972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 266/547] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:52.141733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.153091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 282/548] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:52.206018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.288483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.315305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.315976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef10a91-d858-46f4-a5c3-c942fd0224a0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.352498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fdd48-a6cf-4028-ada7-7d592789f6b9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.352905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.353538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.353817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.354089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.365108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:52.370030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:52.371014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:52.372388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:52.376841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 283/549] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:52.378235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.452337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194bf1-c773-4754-9292-f637978284cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.452953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194bf1-c773-4754-9292-f637978284cd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.453232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194bf1-c773-4754-9292-f637978284cd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.453501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45194bf1-c773-4754-9292-f637978284cd None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.454144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45194bf1-c773-4754-9292-f637978284cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:52.454441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45194bf1-c773-4754-9292-f637978284cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:52.455844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45194bf1-c773-4754-9292-f637978284cd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:52.484980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 267/550] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:52.486445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207824a2-953e-4fbf-8e48-0deaf22146cf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.490154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194bf1-c773-4754-9292-f637978284cd None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.522172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.522532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.522808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.523078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.523743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:52.524174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:52.525504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:52.526956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.538996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 284/551] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:52.541386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.676909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45194bf1-c773-4754-9292-f637978284cd None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.737889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f389673-fa72-48f2-9039-869f8c9225ec None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:52.744468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.749736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.750057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.750263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.763076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:52.767171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:52.793225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:52.794265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:00:52.795849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:52.796365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 268/552] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:52.801175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.812728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:52.813376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:52.813657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:52.813924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:52.822943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.835954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:52.840042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:52.863662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:52.864632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:52.865488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:52.865488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:52.865488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:52.865488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 285/553] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:52.870017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:52.971958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:52.994211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3df9ff-03ee-4cff-9753-7f2a67cd07e3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.030771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abbbb5-a3c1-4b4c-ae04-cc891fccd486 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.066907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.067871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.068343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.068805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.069728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:53.069899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.070510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.070850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.071124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.071364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.071856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:53.072161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.073326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.073582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.086598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 286/554] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:53.091627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.100759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 269/555] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:00:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:53.106231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.199822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee2001-6e63-4674-a9a0-f387063e69ea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.206943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.210617 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.239820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.240828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.241299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.241760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.250209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.251899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:53.264489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:53.265455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:53.266929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:53.268666 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23457003-9c01-4bf1-bcfe-4e6d10d651d6 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.269778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 287/556] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:53.275293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.321783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.322405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.322717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.322989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.413473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.414995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626af687-e9a3-4bb0-997a-4981e063e589 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.481951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.500072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be08e8d-66df-4332-b1df-9ae02c3c2068 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.530684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 270/557] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:53.530684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859a8-c714-46af-acb7-7d4d4b3faa5f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.530684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859a8-c714-46af-acb7-7d4d4b3faa5f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.530684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc859a8-c714-46af-acb7-7d4d4b3faa5f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.530684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 271/558] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:53.536580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc859a8-c714-46af-acb7-7d4d4b3faa5f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.557364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.558294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.558962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.559445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.561513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:53.562338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.564654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15183f9-c5e4-4e14-85cd-76694314738e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.593846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 288/559] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:53.593846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.593846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.593846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.616297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:53.617836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:53.640109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.652012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc859a8-c714-46af-acb7-7d4d4b3faa5f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.688672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.689295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.689576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.689852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.691792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 289/560] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:53.692810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.707886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.715467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-eb8053ea-8f53-4a95-8eac-43d5162ce569 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:53.738429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:53.739500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:53.740422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:53.740422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:53.740422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:53.740422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 272/561] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:53.741202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb246-5f90-4f38-af9b-483d3510b043 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.741385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb246-5f90-4f38-af9b-483d3510b043 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.741708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdb246-5f90-4f38-af9b-483d3510b043 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.746749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 273/562] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:53.748038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb246-5f90-4f38-af9b-483d3510b043 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.768502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.800597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ff95c0-9b09-4d10-85f1-a402a15d88e3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.833933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.834552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.834858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.835130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.844724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:53.846186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:53.848638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdb246-5f90-4f38-af9b-483d3510b043 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:53.863846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:53.864801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:53.866126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:53.866126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:53.866126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:53.866126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 290/563] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:53.866126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:53.877944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:53.890736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:53.891359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.891640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.891914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.901633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-f46190c9-2c3f-496c-a0b3-e1723ba5b2f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:53.906322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:53.907249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:53.908071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:53.908863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 274/564] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:53.910320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408dac-9c1a-4f45-a79f-357b509d983c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.911032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408dac-9c1a-4f45-a79f-357b509d983c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.911315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408dac-9c1a-4f45-a79f-357b509d983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.928051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3d408dac-9c1a-4f45-a79f-357b509d983c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:53.929239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3d408dac-9c1a-4f45-a79f-357b509d983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:53.964280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198191b5-eccf-4fc4-a910-5751167928a6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:53.994846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 275/565] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:53.996547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f040807-7131-4245-aa3d-3c520deec516 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:53.996829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f040807-7131-4245-aa3d-3c520deec516 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:53.997102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f040807-7131-4245-aa3d-3c520deec516 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:53.998383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 276/566] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:53.999583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f040807-7131-4245-aa3d-3c520deec516 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.010828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.011318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.011941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.012225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.012939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.013839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.014175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.016258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.028228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 291/567] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.029018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.094295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.100350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f040807-7131-4245-aa3d-3c520deec516 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.128581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125ecf8-7c69-4c6c-9839-e62cb0af43fa None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.134602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.135211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.135492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.135762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.136396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.136699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.138061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3aafb-e592-434d-af97-59b21adb1a2b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.148877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 277/568] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.150010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.150301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.150598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.162184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.162806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.163086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.163354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.164012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.164309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.165654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.167431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:54.168602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:54.174017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 292/569] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.174807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.229013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.234876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 278/570] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:54.246845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.272039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92d391-1d2e-4f4a-8242-676191df2934 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.300824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.307177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.307796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.308095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.308349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.318732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:54.323706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:54.324701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:54.325746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 293/571] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:54.326739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.337062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f53909-46d8-4b93-9aea-4155ad75f5c2 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.371345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.371904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.372182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.372450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.379456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.380912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:54.392179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:54.393172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:54.394067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:54.397899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 279/572] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:54.398691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.422152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da3d1af-411a-49d7-b5c9-bec53ef06703 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.454509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.455644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.456272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.456518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.456791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.463470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.464899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:54.475585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:54.476466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 294/573] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:54.477355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.500294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690f9c-bbed-45ba-a09f-dba09e34f5a4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.524682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.534421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.535081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.535365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.535883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.546386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.547893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:54.562788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:54.563763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:54.564644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:54.564644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:54.564644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:54.564644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 280/574] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:54.564644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.580997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beace4f-488e-4809-bb91-05703e46c1cb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.612846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.613460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.613744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.614014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.624909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:54.629869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:54.630799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:54.631819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 295/575] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:54.632755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.665150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549defc-5755-4d4e-9db6-ec5b74aca586 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.675668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.698438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.699115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.699432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.699746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.700471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.700822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.702809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.710594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 281/576] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.711204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.735710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65438c-647e-46c9-ae89-522d740b8a8b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.745813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.769165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.769793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.770074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.770352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.771093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.771404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.772801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.785131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 296/577] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.786465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.808987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c886aa-a2f6-4c4d-ac1e-d6e0b6c42a3e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.841437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.842027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.842343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.843157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.843839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.844141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.845551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.857033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 282/578] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.857819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.887436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.892192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ddf19-5cfe-4b05-aed7-631977c9a021 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.925412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.926009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.926284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.926552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.927218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.927518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.928898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.939887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 297/579] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.940493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:54.954992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f050400-ac2b-4806-9d14-31c68b514d0e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:54.973445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:54.987690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:54.988289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:54.988571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:54.988841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:54.989491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:54.989791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:54.991186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:54.999036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 283/580] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:54.999798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.042314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4636918c-85a2-4c4d-ba6a-d99ac81d9a0d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.074936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.075552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.075830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.076099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.086056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.087522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.090348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.098119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234fbb0b-326e-4d2f-8750-936989c15fc3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:55.101450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:55.102458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:55.103386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:55.103386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.103386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:55.103386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 298/581] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:55.103618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.133794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.134390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.134700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.134974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.135697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=70fc784da233409a9fa518088a4ac68d) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:55.136007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.137422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.142845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 284/582] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:00:54 2022] GET /identity/v3/role_assignments?user.id=70fc784da233409a9fa518088a4ac68d&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:55.143551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.188293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.210093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c3809-ecb6-4832-8294-dc76ad38831d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.249427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.250099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.250387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.250691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.266976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=1561e7c10ad04eeda2e31c3e31cf3c32, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.268017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b739b0-4d96-4a03-942f-edfac8a42330 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.268461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.287816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 299/583] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:55 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:55.287816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.318797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.319719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.320085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.320427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.329207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.330812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:55.344000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:55.344999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:55.345962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 285/584] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:55.346549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.369264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.414995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6127a99-0df3-4b74-9bd3-8bb9f3db5a5e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.450439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.457217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.458081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.458662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.459080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.467120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d2c1-deac-46a9-98d0-35ec75a85e0c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.472331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.473980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:55.491152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:55.492296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:55.493381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:55.493381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.493381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:55.495192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 300/585] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:55.496233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.504204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.504818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.505098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.505370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.506021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:55.506667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.507781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.521086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 286/586] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:55.522154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.608591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0c5f4c-5f09-4164-88dd-4bd3eb028f01 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.609724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.634031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e03f88-f76d-4c21-b760-1907fcc28c59 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.643562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.644166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.644445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.644713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.645838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:55.646149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.647585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.666165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 301/587] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:55.666915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.681014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.682085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.682435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.682811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.683524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:55.683893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.685463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.698218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 287/588] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:55.699159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.711107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.768767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f43e-d369-4c4b-bdac-1e6ef28f69f6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.804473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.805096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.805451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.805631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.809480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed441990-0634-4fdb-8bb3-477619533d39 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.813574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.815037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:55.827108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:55.828260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:00:55.829429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:55.832855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 302/589] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:55.833680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.851243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.852113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.852556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.852959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.861192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.867472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.869114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:55.883852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:55.884989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:55.886010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:55.886010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:55.886010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:55.891977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 288/590] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:55.892946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:55.949905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9cede-39d7-4557-ba8d-caea07882b64 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:55.951409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:55.985470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:55.986339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:55.986810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:55.987236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:55.995184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:55.996769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:55.998565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0d667-68c8-4e0b-89c9-648b20e09aa7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:56.008280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:56.009400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:00:56.010482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:56.012525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 303/591] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:56.013549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.034186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.034829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.035108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.035378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.036355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.036657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.038175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.061394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 289/592] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:00:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.061394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.110142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.122315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802967ec-3a63-4357-b7e1-4fcf3e8b000a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.157914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.158817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.159238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.159649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.160435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.160875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.162647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.166437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb2094c-445e-45cf-aad1-ef3b9dae3016 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.176035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 304/593] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.177137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.205860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.209526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.210459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.210590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.210877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.211566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.211903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.213892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.232597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 290/594] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.234016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.288117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446f4593-8945-44bc-9840-5ac2cc0b5340 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.324098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.325001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.325418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.325825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.337032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.338682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.341524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd8e5a-76f8-4c2a-b0b9-03c02d1a2732 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.350433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:56.354807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:56.355916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:56.356988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:56.356988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:56.356988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:56.362960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 305/595] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:56.363976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.380386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.381017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.381295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.381562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.388696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.390776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:56.402320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:56.403422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:56.404487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:56.408628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 291/596] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:56.409465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.445881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.472512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8ce28-1e72-4c69-9d09-3caedea6fd7a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.506371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.507024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.507301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.507570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.518472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472d55a-3c10-4871-a481-123788b9537c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.519044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.520600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:56.537447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:56.538662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:56.539589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:56.539589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:56.539589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:56.539589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 306/597] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:56.539823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.559852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.560735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.561154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.562156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.563029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.563608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.565142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.578724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 292/598] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.579764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.608577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.650553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a39f5b-84a3-45f9-9292-71b654a16374 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.675884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.685679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.686810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.687152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.687621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.688101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.688404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.689975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.690840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbb262-d2b7-4f6c-98d9-641dc45dc971 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.703195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 307/599] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.704191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.730493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.731157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.731443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.731717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.732463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:56.732771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.734164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.751443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 293/600] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:56.752338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.809054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093de2-4f0b-4392-9b6c-9980d5976df7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.843624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.854207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.855192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.855701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.856139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.859518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e587a-797d-49e2-9ba9-fbe7a1cb4e2c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:56.868091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.869569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:56.887832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:56.888756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:56.890310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:56.896921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 308/601] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:56.896921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:56.917240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167692f9-9849-458c-806c-720188413a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:56.918117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167692f9-9849-458c-806c-720188413a62 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:56.918537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167692f9-9849-458c-806c-720188413a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:56.919516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167692f9-9849-458c-806c-720188413a62 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/eacd5dcb5de7474da98c2e7f30a4e6db/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:56.940142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167692f9-9849-458c-806c-720188413a62 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=eacd5dcb5de7474da98c2e7f30a4e6db, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:56.942286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167692f9-9849-458c-806c-720188413a62 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:56.946704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:56.961409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 294/602] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:56 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/eacd5dcb5de7474da98c2e7f30a4e6db/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:56.962238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167692f9-9849-458c-806c-720188413a62 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.039095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c2ddb-c65d-4b52-92e9-beb4993f53ca None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.091484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167692f9-9849-458c-806c-720188413a62 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.131950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.132653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.132918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.133193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.137271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.138210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.138738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.139301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.141137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:57.141699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:57.143527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:57.147641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:57.155160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:57.157205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:57.166414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 295/603] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:57.172835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:00:57.184208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:57.185150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:57.186022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 309/604] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:57.191710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.285749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc26b0-13d4-4e3a-97ed-e3b660b46c7b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.321965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.323065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.323550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.324064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/5882280742154859b52fa47c350a137f/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.335981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a354131-4bcf-4449-9c3b-2d8a94cb5d81 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.340503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=5882280742154859b52fa47c350a137f, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:57.342167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64deba26-af31-44ca-8f1e-b4596e2a23cd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:57.345080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:57.360641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 296/605] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:57 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/5882280742154859b52fa47c350a137f/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:00:57.380504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cfd-56d5-4677-8fa8-b8f8eae49635 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.381185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cfd-56d5-4677-8fa8-b8f8eae49635 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.381709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef906cfd-56d5-4677-8fa8-b8f8eae49635 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.383894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 297/606] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:00:57.391588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902967-7d50-4a6a-85d1-ad1b80a4888e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.392199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902967-7d50-4a6a-85d1-ad1b80a4888e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.392752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae902967-7d50-4a6a-85d1-ad1b80a4888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.395418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.396055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.396343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.397325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.398113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:57.398429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:57.399900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:57.418910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 310/607] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:57.420344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ae902967-7d50-4a6a-85d1-ad1b80a4888e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:57.421860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ae902967-7d50-4a6a-85d1-ad1b80a4888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:57.423784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.476326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:57.511477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 298/608] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:00:57.516633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975dea7-b943-4d03-beff-63b7b81cbdf7 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.516779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975dea7-b943-4d03-beff-63b7b81cbdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.517042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0975dea7-b943-4d03-beff-63b7b81cbdf7 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.523826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 299/609] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:57.525038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.525319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.525588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.558622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:57.561078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:57.587307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6810ac-d113-4786-b47e-a71246e3525f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.633255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.634054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.634483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.634802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/5c5297e9a66d4124bc4888e5a92dc885/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.646233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 300/610] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:57.647114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.653645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=5c5297e9a66d4124bc4888e5a92dc885, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:57.655116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:57.687026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 311/611] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:00:57 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/5c5297e9a66d4124bc4888e5a92dc885/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:00:57.688052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.740436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:57.776992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e5814-16f2-42c4-88df-c441c83e2dd6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.786988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:57.819916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:57.820798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:57.821220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:57.821631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:57.834993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:57.840253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:57.841324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:57.842371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:57.845644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 301/612] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:57.851115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:57.958508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992658d3-0d4e-4c77-b16b-c6d47ef59a61 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:57.980459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7386b59-331a-4a28-b5f0-fbe163d9ebbd None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.011796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.012914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.013501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.014257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.023437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.025841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.026139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.026485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.027208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:58.027512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.027726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:58.028887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:58.033845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:58.034963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:58.035927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:58.038652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 312/613] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:58.043059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.043945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 302/614] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:00:58.050000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.115027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:58.131943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:58.157741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68881861-bbe1-47bb-82af-ba4ca1fac62e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.197568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.198595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.199063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.199442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.200240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:58.200654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.203842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.212659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 313/615] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:58.227995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.304481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70f7097-0586-46be-ac72-eb52a56d36f0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.341412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfcae8-b145-47fc-b156-e0af3e44d3d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.363476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.364379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.364796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.365248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.376824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.378518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:58.393675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:58.394787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:58.395779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:58.395779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:58.395779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:58.397546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 303/616] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:58.403484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.413210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.413844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.414175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.414448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.421667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:58.425538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.427219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:58.441979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:58.443049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:58.444268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:58.444268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:58.444268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:58.444268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 314/617] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:58.450410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.575091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:58.626681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1734f6-702c-4d5e-afca-0f4779201744 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.631365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dfe677-ee07-48c5-a502-a11bf76344a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.669990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.670748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.671567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.671843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.672492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:58.672797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.674162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.694123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.694804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.695089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.695362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.696069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:58.696377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.697754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.702222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 315/618] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:58.707766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.726636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 304/619] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:58.732331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:58.801856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:58.896738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15380886-4a1d-4bce-baab-a03333775c4c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.919440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061bdc14-22cb-437b-b419-aff2060da23c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:58.964423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.965054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.965334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.965605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.979098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.980536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:58.986533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:58.987195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:58.987478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:58.987755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:58.995829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:58.997278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.000367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:59.004031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:59.005335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:00:59.006642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 316/620] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:00:59.012745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:00:59.013286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:59.014293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:00:59.015444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:59.017462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 305/621] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:00:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:59.024400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.164062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442159-1714-48fc-81e8-a86e91970264 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.183090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.200822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.201432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.201713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.201989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.203313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:59.203658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.205078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.224250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 317/622] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:59.230262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.241688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa676783-f96d-48c9-ad86-5c9a0f585660 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.335654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442423f8-6b07-4a70-a274-7fe702bf903d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.347025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.348257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.348705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.349114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.349987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:59.350530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.352229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98be61f-e775-4ba8-81de-b8c7d081571b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.371217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.371730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 306/623] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:59.371963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.373361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5cb0c-a727-4ae2-84c6-f0122d873ed1 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.373770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5cb0c-a727-4ae2-84c6-f0122d873ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.374279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5cb0c-a727-4ae2-84c6-f0122d873ed1 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.379733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.379733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.379917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 307/624] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:59.379917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5cb0c-a727-4ae2-84c6-f0122d873ed1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.418819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.457863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.460017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15396420-3e75-461c-ba44-5a93998ce43d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.466782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.522466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5cb0c-a727-4ae2-84c6-f0122d873ed1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.559435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.560981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.561462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.562639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.573777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.575522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.585706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 318/625] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:00:59.585706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.585706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.585706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:59.596475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:59.597585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:59.598474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:59.598474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:59.598474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:59.600632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 308/626] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:59.601576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.606594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:59.607984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:59.675765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.682033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 319/627] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:00:59.683736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.720986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb6c9f-4696-4375-bbb8-118d9fb6c8d4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.732048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.758593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.759238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.759516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.759785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.771384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.772816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-f05315bb-d3ed-4cda-a632-a86818dca2fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:00:59.787137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:59.788112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:00:59.793883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:00:59.793883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:00:59.793883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:59.794055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e4cf-a9a7-460c-89b0-064c9edefdc0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 309/628] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5735-c3a0-48e0-84db-4e6e0f91488f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5735-c3a0-48e0-84db-4e6e0f91488f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b5735-c3a0-48e0-84db-4e6e0f91488f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 310/629] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:00:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:00:59.794137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b5735-c3a0-48e0-84db-4e6e0f91488f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.830338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.830981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.831257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.831525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.832164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:59.832465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.833815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:00:59.846886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 320/630] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:00:59.847663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:00:59.910342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b5735-c3a0-48e0-84db-4e6e0f91488f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.912499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:00:59.947636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.948317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.948603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.948814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.957213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b69cdf-44d2-451b-9193-6bb4ec422d67 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:00:59.959743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-887ff8af-f7ab-45d9-bc0c-ea818ecc69e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:00:59.964427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:00:59.965517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:00:59.966548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:00:59.972641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 311/631] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:00:59.973402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.973706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.973965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.991385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:00:59.992545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:00:59.994663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:00:59.995275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:00:59.995550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:00:59.995817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:00:59.996453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:00:59.996751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:00:59.998340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.011321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 321/632] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.011568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.012387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.070614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 312/633] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:00.070614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.124741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387026c-17f6-4008-adcc-1715d1310a3f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.141699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.158969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.159591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.159879 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.160148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.166843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.168267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.168390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4f7f6-a5d6-4c9d-b2dc-d617aaf032f1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:00.179113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:00.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:00.181057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 322/634] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:00.181553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.204102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.204644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.204926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.205196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.205850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:00.206152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.207536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221c238-3447-4b95-bba7-46ccd97c77aa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.215204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 313/635] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.216324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bda9-3c36-412c-96b2-baacf68891ee None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.216611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bda9-3c36-412c-96b2-baacf68891ee None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.216884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447bda9-3c36-412c-96b2-baacf68891ee None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.218805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 314/636] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:00.223218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447bda9-3c36-412c-96b2-baacf68891ee None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.238842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.277868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8785-dcbe-4fd8-8b6b-c2ac8eb46ec2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.312026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.312639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.312919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.313186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.319729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.321143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.322661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447bda9-3c36-412c-96b2-baacf68891ee None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:00.331694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:00.332725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 323/637] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:00.333612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.361002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.361625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.361907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.362182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.371050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.372158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:00.377641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:00.378740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 315/638] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:00.379604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.432977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac91065-61d8-4fd1-9bd0-5e165666e757 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.451625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.466557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.467123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.467404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.467676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.468854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:00.469157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.470545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b9431-e274-425f-88f9-cfd163e2b7d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.475732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1191b-4e12-4133-9347-7ceb52496488 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.481565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 324/639] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.482726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022bc-d611-4774-a104-47293227d401 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.483009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022bc-d611-4774-a104-47293227d401 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.483281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7022bc-d611-4774-a104-47293227d401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.500312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4a7022bc-d611-4774-a104-47293227d401 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:00.501525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4a7022bc-d611-4774-a104-47293227d401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:00.512498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.513115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.513393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.513660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.523265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.524676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:00.539547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:00.540540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:00.541450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:00.541450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:00.541450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:00.541450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 316/640] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:00.541698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.567813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 325/641] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:00.568931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7022bc-d611-4774-a104-47293227d401 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.595433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.648857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b35b24c-3d59-4d1d-9ffb-7e04ad3c91c8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.664037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7022bc-d611-4774-a104-47293227d401 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.673378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.682143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.682775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.683059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.683334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.684023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:00.684329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.685742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.697084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.697660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 317/642] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.697785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.697969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.698199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.698404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.698954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:00.699259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.700605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.708954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 326/643] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.709722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.795855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.801125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea8427-4418-43b5-b1f5-5d9f32d32ad0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.805324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d890f8f6-bb36-4759-a351-63f2b27ac09f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.834993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.835633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.835911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.836185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/70fc784da233409a9fa518088a4ac68d/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.837905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:00.838502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:00.838801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:00.839769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:00.839769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:00.839992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.841366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.851253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=70fc784da233409a9fa518088a4ac68d, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:00.852337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:00.854535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 327/644] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:00.859529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.864740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 318/645] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:00 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/70fc784da233409a9fa518088a4ac68d/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:00.865548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:00.907406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:00.979151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d9c847-102f-4190-850c-cdec285f2e15 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:00.980704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62589d3-df2a-4ba4-a2aa-af2e61c4a710 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.019484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.020224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.020511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.020883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.021611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:01.021918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.022023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.023679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.023679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.023679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.024526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.030129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 319/646] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:01.031043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.035564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:01.040996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:01.042009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:01.042970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:01.048338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 328/647] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:01.048338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.071458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.175993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.180357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e290b72b-682e-440b-b43b-7667e5b1ebbe None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.203154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a17-70a6-43d8-942c-0f30331e1f88 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.226909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.227548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.227830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.228107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.240251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.241808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.244021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.244775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.245050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.245353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.254525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:01.255767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:01.256676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:01.258120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:01.258120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:01.258120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:01.258120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 320/648] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:01.258120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.259153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:01.279261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:01.280275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:01:01.281785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:01.284825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 329/649] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:01.285859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.378355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.392479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e6b97-fb20-4ae6-a6d9-8c425de6850d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.414003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f0ddb-1802-4c14-bcd3-d096893abc3c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.440075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.441091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.441822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.442495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.443333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:01.443966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.446068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.453271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.453997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.454378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.454779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.455526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:01.455932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.457263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 321/650] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:01.457603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.458463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.471422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 330/651] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:01.476736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.477703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.585376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37692d-31db-42d3-b505-9f3a3b96d006 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.589112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da19ce4-a94e-402a-ad0a-d2a115d85d6d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.623101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.624086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.624496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.624894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.625725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:01.626133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.626619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.629100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.634285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64448703-4058-46e3-9219-477ce7364427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.635206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64448703-4058-46e3-9219-477ce7364427 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.635654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64448703-4058-46e3-9219-477ce7364427 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.636067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64448703-4058-46e3-9219-477ce7364427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.649594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64448703-4058-46e3-9219-477ce7364427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.654526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 322/652] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:01.654526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.657642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64448703-4058-46e3-9219-477ce7364427 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-64448703-4058-46e3-9219-477ce7364427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:01.676838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:01.677968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:01.679093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:01.679093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:01.679093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:01.681240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 331/653] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:01.682779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64448703-4058-46e3-9219-477ce7364427 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.737367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.770688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f028f1-6b74-4251-ba15-94ea70449a93 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.814740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64448703-4058-46e3-9219-477ce7364427 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:01.814966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.814966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.815176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.815249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.825742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.828767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:01.844173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:01.845356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:01:01.846516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:01.853291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 323/654] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:01.854303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:01.854507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:01.855160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:01.855542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:01.855927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:01.926629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:01.953981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:01.956235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:01.971146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78208f6a-6c47-48ae-ab05-c4213b12ed9d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.017453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.024416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.025278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.025598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.025990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.026769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:02.027175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.028754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.041564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 324/655] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:02.042474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.075523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 332/656] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:01:02.077254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.166046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.181780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2a2c1-6c63-4606-91aa-b13930031e7e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.186757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4e4d3-d60f-46e3-ad17-602e4ab3f23c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.222350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.223144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.223612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.223979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.226878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.228102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.228529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.228941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.229812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:02.230280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.232077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.232473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.234646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:02.246374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:02.247570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:01:02.248721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:02.250760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 333/657] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:02.251614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.254479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 325/658] 10.222.0.49 () {56 vars in 1137 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:02.256018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.310097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.366788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49802f-ecb5-44cb-a43e-6fe5bb978704 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.371638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f959f4-e052-4c03-822a-9290e76bd870 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.402623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.403259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.403545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.403829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.415461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.415461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.415461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.415461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.416127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:02.416127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.416291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.416852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.416956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.418854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:02.434013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:02.435135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:02.436037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:02.436037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:02.436037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:02.436037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 334/659] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:02.436259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.441361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 326/660] 10.222.0.49 () {56 vars in 1139 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:02.442387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.550341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e3ee79-4c91-44db-9e25-5afab6a1d8ed None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.576711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf4c4ef-ef5f-4d06-abb3-23220abd20f1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.587388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.588454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.589148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.589525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.590295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.615854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.616813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.617319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.617790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.618664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:02.619169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.620811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.636309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 327/661] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:02.637464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.674221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.684164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.685871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.773683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca1882-5552-4b80-86dd-7b2caed37473 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.798853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 335/662] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:01:02.800057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:02.819106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.819739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.820262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.820544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.866558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.906881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:02.916100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.917538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4402ee0-a482-4354-95d7-a26dc72ab296 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.922452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec43bbae-d013-4b7b-a4e9-31c9c761ff72 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:02.961006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:02.961637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:02.961921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:02.962203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:02.969088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:02.970540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:02.982784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:02.983709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:02.985134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 336/663] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:02.985625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.034044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 328/664] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:01:03.036349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d19-6030-4eaa-8d5a-468fdd31f70f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.036713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d19-6030-4eaa-8d5a-468fdd31f70f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.037065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57267d19-6030-4eaa-8d5a-468fdd31f70f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.039407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 329/665] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:03.040209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267d19-6030-4eaa-8d5a-468fdd31f70f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.089548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc90cc-7dfe-422e-94d7-4dd4acd283ee None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.111274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.127288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.127917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.128193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.128467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.138392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.139849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.151671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57267d19-6030-4eaa-8d5a-468fdd31f70f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:03.153794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:03.154775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:03.155677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:03.155677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:03.155677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:03.155677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 337/666] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:03.155677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.167757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.197558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.198160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.198439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.198742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.199526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:03.199837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.201244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937066cb-e27e-4179-bd0a-57c326a6707b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.219366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 330/667] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:03.219366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.219366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.219366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.233859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:03.235111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:03.262793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b47ce9-fd74-41a4-a0b0-48d21a3bdb15 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.295628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.296240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.296516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.296810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.299113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 331/668] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:03.299919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.320707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.322144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e492003a-ae0c-4c21-8625-dd5f77df21b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:03.337112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:03.338209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:03.339156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:03.339156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:03.339156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:03.339156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 338/669] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:03.339418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce776-8c90-4cd7-a896-e8c3bb40caaf None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.339985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce776-8c90-4cd7-a896-e8c3bb40caaf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.339985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7ce776-8c90-4cd7-a896-e8c3bb40caaf None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.341193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 339/670] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:03.341770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce776-8c90-4cd7-a896-e8c3bb40caaf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.357863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.399134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849888f-286c-40a0-8333-0af90200ec7a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.401907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.434318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.435382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.435884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.436355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.438063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:03.438888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.440750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.444678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7ce776-8c90-4cd7-a896-e8c3bb40caaf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.456731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 332/671] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:03.457535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.481880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.482413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.482740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.483273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.483738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=eacd5dcb5de7474da98c2e7f30a4e6db) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:03.484048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.486008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.491592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 340/672] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/role_assignments?user.id=eacd5dcb5de7474da98c2e7f30a4e6db&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:03.492352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.582350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a38bf-f109-484a-8e42-b92c903126b8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.599934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2bce-a148-44b6-8074-48adf1e0f6ab None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.601906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.621421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.622192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.622472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.622780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.625666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.636978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:03.638917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.639538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.639817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.640094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.641308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:03.641613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-36e5f1ca-335e-4a9f-a3db-7f4f6f51718b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:03.642697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:03.643671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:03.644538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:03.645204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.645275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 333/673] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:03.646761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.647054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.647337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.657171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 341/674] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:03.657336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.667043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:03.668216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:03.748112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 334/675] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:03.749287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.779863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c37df-0d68-45b4-a7ee-d49ef40bbcaa None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.815530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.816158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.816441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.816714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.823393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.824838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:03.835901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:03.836805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:03.837712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 342/676] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:03.838262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.876084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8684a-b7d6-4ac9-bc8f-650bb03b6a9b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.885689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.886338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:03.910975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.911580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.911865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.912150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.913050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:03.913349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.914734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:03.923113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 335/677] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:03.923886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:03.951324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86acb905-d1f4-4137-93f2-d85c36089755 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:03.985499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:03.986296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:03.986425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:03.986673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:03.993476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:03.995261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:04.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:04.010724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:04.011638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:04.012076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 343/678] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:04.012533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.035152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8bd3d-ac8a-435d-8203-dfb1e21cc32b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.070233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.070891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.071171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.071517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.082211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:04.087432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:04.088343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:04.089172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 336/679] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:04.089866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.119503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.126234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d48c8-5f44-4849-a542-3aa0a9dcfe8b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.128109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.161999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.163155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.163448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.163725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.164458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.164757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.166170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.177947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 344/680] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.178774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.201557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa667-1b71-4679-8c0e-631494dc29f0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.284435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac3bd7-f426-4a3b-ad0f-1ad11b5ed7cc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.320182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.320796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.321079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.321348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.322045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.322350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.323814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.336578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 345/681] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.337411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.363878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.367302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.368141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.368425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.368706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.378894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.380351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:04.394690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:04.395611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:04.396527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:04.396527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:04.396527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:04.396527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 337/682] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:04.396527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.449531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542d43d-aeb8-45f4-97c9-e08f6e73cf85 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.482920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.483404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.483762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.484050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.484324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.485508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.485814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.487294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.499906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 346/683] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.499906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.507457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f786b3-97bc-4dde-bc3d-732070ef7088 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.542616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.543479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.543935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.544401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.545702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=5882280742154859b52fa47c350a137f) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.546216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.548758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.556504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 338/684] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/role_assignments?user.id=5882280742154859b52fa47c350a137f&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.557478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.601503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.626842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2e775-2de5-4492-9f07-410d02ff10ea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.669196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.669506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.669956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.670314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.671206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=5c5297e9a66d4124bc4888e5a92dc885) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.671589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.673707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.680723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 347/685] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/role_assignments?user.id=5c5297e9a66d4124bc4888e5a92dc885&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.681755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.703837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c6645-dd52-436c-887a-fd68c158b184 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.754915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.755965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.757248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.757747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.758697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:04.759311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.761250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.777908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 339/686] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:04.784503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.795916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.811817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b78e3d-6d38-4d29-8675-8e18c22f462b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.877256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.877903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.878186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.878461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.897888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.900314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:04.924835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:04.925916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:04.926988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:04.926988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:04.926988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:04.926988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 348/687] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:04.927566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:04.931579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:04.932565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3019bc-a1c0-41b0-a1f2-5260a5bad03b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:04.979304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:04.979936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:04.980217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:04.980489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:04.989521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:04.990992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:05.005077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:05.006266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:05.007450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 340/688] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:05.013066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.067880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9584729-d929-4fc9-86ff-4c5706ad9199 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.095215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:05.150603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf570c-b5c9-426d-a093-e5161ab8765b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.150995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.150995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.150995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.150995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:05.151536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:05.151536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:05.153982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.185801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 349/689] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:05.192295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.213077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.213749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.214029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.214299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:05.215263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:05.215568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:05.219170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.247005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 341/690] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:05.252802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.308419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:05.379223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbe12c-87cc-4fe7-8179-d89c0d8f2bc8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.441792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.442424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.442721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.445155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:05.447352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a8ed5-5ef1-41a7-a913-4ce96bae61c4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.465605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:05.467641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:05.490062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:05.491874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:05.493682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:05.500368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 350/691] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:05.501254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.506928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:05.522336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.522986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.523255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.523576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:05.648071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:05.649681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.689231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ab807-be34-49d3-b52f-584f9f390264 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.716854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:05.759572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.760252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.760540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.760816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:05.761563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:05.761869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:05.763468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:05.820211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 351/692] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:05.826488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.843688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 342/693] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:01:05.855641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:05.907286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:05.949488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae248ff-1388-4872-a87f-d5cdbfe946b6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:05.994710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:05.995711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:05.996233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:05.996804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.016478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a92bc-b2ac-4de9-b46e-9447f7ce89aa None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.051930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.052501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.052778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.053056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.064938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.066366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:06.083049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:06.084019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:06.084897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:06.084897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.084897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:06.084897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 343/694] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:06.088583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.091551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.114984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.116695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e567d48-9953-445f-ba33-7d6acecf326d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.169389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.192920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57b0ce-386b-4a3d-b5d2-be202f65e728 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.222755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 352/695] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:01:06.224353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a98fd-ab4e-4239-a972-04de6d496df3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.224850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a98fd-ab4e-4239-a972-04de6d496df3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.225342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a98fd-ab4e-4239-a972-04de6d496df3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.226986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.227239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 353/696] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:06.228159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a98fd-ab4e-4239-a972-04de6d496df3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.231309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.231601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.231935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.233135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:06.233135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.234842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1b89e-1342-410e-8029-bad9fb610146 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.248431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 344/697] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:06.249691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.249976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.250326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.271399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:06.272568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:06.313699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.346741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a98fd-ab4e-4239-a972-04de6d496df3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.352504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 345/698] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:06.353634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.401532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.402660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.403195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.403667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.415772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.417521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:06.433591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:06.434547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:06.435507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:06.435507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.435507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:06.435678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.437866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 354/699] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:06.438943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.482203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94301c0c-fa51-4d06-a686-62888fba2aea None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.517384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.518015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.518279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.518553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.525559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.527029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:06.539608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:06.540539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:06.541428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 346/700] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:06.542016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.553002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963943cf-896d-4dde-b0e1-732bc9b32787 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.583066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.592720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.593671 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.594864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.595359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.606888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.608608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:06.624281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:06.625289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:06.626235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:06.626235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:06.626235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:06.628320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 355/701] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:06.629333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.671378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd495323-6acb-4dc2-aa1c-7ed1e348cdce None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.698633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.710157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.710793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.711074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.711344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.711986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:06.712289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.713656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.726841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 347/702] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:06.727727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.738795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d690e-407a-4e77-a4f0-5f24a588f28a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.775200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.776292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.776775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.777334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.778259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:06.778809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.780672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.795461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 356/703] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:06.796603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.838877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8d7d9-b1da-44d5-b7a7-2b9e9f578f03 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.846966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.878092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.878963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.879359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.879740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.880787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:06.881112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.883187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.897101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 348/704] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:06.898005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:06.921272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8046a62-bb47-4af5-9a4e-63907e4b10ef None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:06.947167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:06.961665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:06.962656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.963148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.963517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.970260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:06.971719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-bf20ae2c-7f78-4a66-8b40-05ec76f5f184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:06.982641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:06.983609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:06.984605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 357/705] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:06.986127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9859-0577-48d1-92d5-60f920af1cbc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:06.986127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9859-0577-48d1-92d5-60f920af1cbc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:06.986127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f9859-0577-48d1-92d5-60f920af1cbc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:06.990622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 358/706] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:06.990622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f9859-0577-48d1-92d5-60f920af1cbc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.007376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da483b18-968b-4df0-9670-2a5f8d0d74be None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.041737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.042344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.042651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.042927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.043668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=70fc784da233409a9fa518088a4ac68d) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:07.043979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.045432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.058610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 349/707] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:06 2022] GET /identity/v3/role_assignments?user.id=70fc784da233409a9fa518088a4ac68d&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:07.058610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.133643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f9859-0577-48d1-92d5-60f920af1cbc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.136327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:07.189087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.190112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.190401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.191046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.191848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.194243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16946a01-761d-4329-b822-827b7252bd27 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.196076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aac880-c6aa-4690-925d-35805225443a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.211865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:07.226559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 359/708] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:01:06 2022] POST /identity/v3/users => generated 254 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:07.227892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493a68-1f5e-4beb-99c2-0d5a46d43027 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.228188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493a68-1f5e-4beb-99c2-0d5a46d43027 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.228462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4493a68-1f5e-4beb-99c2-0d5a46d43027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.245346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.246312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.246836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.247318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.248228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:07.248772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.250046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a4493a68-1f5e-4beb-99c2-0d5a46d43027 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:07.251249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.252033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a4493a68-1f5e-4beb-99c2-0d5a46d43027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:07.267572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 350/709] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:07.274090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.353734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 360/710] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:07.376024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf71ae-aa5b-4a1d-a7b5-221c95dab049 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.376024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf71ae-aa5b-4a1d-a7b5-221c95dab049 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.376024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf71ae-aa5b-4a1d-a7b5-221c95dab049 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.378870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 361/711] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:07.381164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf71ae-aa5b-4a1d-a7b5-221c95dab049 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.426213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a922538-c1cd-41a0-9380-0d797a5f0b4b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.473756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.474299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.474599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.474876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.475601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:07.475909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.477378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5216db43-bc1d-4a9a-a1cc-8aa0454cf85d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.483206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 351/712] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:07.486642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.486642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.486642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.506880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:07.508093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:07.517154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:07.524063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf71ae-aa5b-4a1d-a7b5-221c95dab049 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.541508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:07.574506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.575218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.575504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.575779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.577337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.578897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:07.592854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:07.593968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:07.594909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 362/713] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:07.600288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.612225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 352/714] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:07.645704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.778116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e336a4-391f-4de3-a577-8853522efcb7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.799585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dff1e-b5e0-4e2c-85fb-730842adfc11 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:07.866607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.868085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.869590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.870492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.871669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:07.872690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.873514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:07.875146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.876021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:07.876491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:07.876963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] PATH_INFO: `/v3/services/network` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:07.878908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:07.880472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:07.889005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 363/715] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:07.894600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:07.896650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:07.897615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:07.898451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:07.907821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 353/716] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/services/network => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:07.909067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:07.941013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:07.954389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:08.082839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f11725-25c0-4f65-a617-57443cd0e51c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:08.097170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab08096-f42b-41fa-a72f-6dd303da8deb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:08.147532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:08.147775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:08.148135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:08.148412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:08.148515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:08.148686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:08.148796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:08.149038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:08.149306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:08.149610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:08.149718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:08.149969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:08.150987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:08.151342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec0574-ea5c-402a-948d-7eb8b9f392c1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:08.157860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 364/717] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:08.163043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 354/718] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:08.169011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:08.282985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:08.303579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1196b46f-4e8a-4c72-a4ab-956969cb373e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:08.341388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:08.342077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:08.342361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:08.342651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:08.343304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] RBAC: Adding query filter params (type=network) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:08.343602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:08.345026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e64cc-9b68-4866-a209-8adc59000393 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:08.350629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 365/719] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:08 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:09.636670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb04f0-5371-46df-9709-ec0a9faa3e74 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.637654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb04f0-5371-46df-9709-ec0a9faa3e74 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.638122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb04f0-5371-46df-9709-ec0a9faa3e74 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.639932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 355/720] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:09.645327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.646119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.646639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.669289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:09.670797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:09.749723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d919ab-0e15-4034-b76f-136d12fceaec None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.750063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d919ab-0e15-4034-b76f-136d12fceaec None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.750275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d919ab-0e15-4034-b76f-136d12fceaec None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.752425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 356/721] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:09.759763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 366/722] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:09.762206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b2f6-cad3-4811-9044-5fa987fd6875 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.762781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b2f6-cad3-4811-9044-5fa987fd6875 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.763269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41b2f6-cad3-4811-9044-5fa987fd6875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.783253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:09.792057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cd41b2f6-cad3-4811-9044-5fa987fd6875 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:09.793248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cd41b2f6-cad3-4811-9044-5fa987fd6875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:09.891037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:09.894505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 357/723] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 11:01:09.901711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5271e-0413-4666-af8a-958fd8cd2a53 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.901711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5271e-0413-4666-af8a-958fd8cd2a53 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.901711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b5271e-0413-4666-af8a-958fd8cd2a53 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.901711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 358/724] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:09.903182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.903466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.903744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.922728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:09.924444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:09.926489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc508b-e9ba-46a8-94ff-a326d40e28e6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:09.944030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:09.969101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:09.970199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:09.970745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:09.971211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:09.981949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:09.987164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:09.988080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:09.989002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:09.991762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 367/725] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:09.992806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.000495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 359/726] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:01:10.001878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.097400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210b34e-c076-425a-a751-59a5efae3b92 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.116080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282cb48-6082-4cdb-b91a-c5be8bae2d4f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.132773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.133403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.133705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.134107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.150969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.152675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:10.153906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.154772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.155264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.155746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.156603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:10.157094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:10.158794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:10.159718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:10.160551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 368/727] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:10.161269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.161857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.173779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 360/728] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:10.175272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.194735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.287638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28ee9b-9f02-4caf-b89f-d6647a56a9a5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.299596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd727d36-dd5a-4512-8f01-cf81c247a647 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.331183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.332318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.332672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.333222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.337051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.337755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.338106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.338391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.339139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:10.339446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.341010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.344781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:10.349818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 361/729] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:10.350664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:10.351266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:10.352200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:10.353037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:10.358611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 369/730] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:10.358611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.433902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.437549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.465460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d817-838a-4d45-b7b7-945074dbabbb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.496494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c8848-a8e9-43f3-a4da-b95f0aeb582c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.500687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.501947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.502363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.502768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.513850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.515448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:10.530730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:10.531731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:10.532689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:10.532689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.532689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:10.532689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 362/731] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:10.532689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.549342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.550160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.550450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.550807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.568891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.571710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:10.592870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:10.595278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:10.596881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:10.596881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.596881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:10.596881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 370/732] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:10.596881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.658697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289a424-30a2-4b07-b1bb-89207f664d23 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.688288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.698934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.699547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.699829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.700104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.700885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:10.701193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.702663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.713119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fd884-8f3e-4f0c-bf2a-7e83545d56f7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.717168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 363/733] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:10.718108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.721408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.750286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.751469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.751762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.752037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.752688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:10.752992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.754379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.767229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 371/734] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:10.768073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.840321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a40e-7f9a-4fa5-9b28-182fc2573b09 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.873554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.874161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.874438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.874739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.875371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:10.875676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.877595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.880081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cfdc9e-81d4-44a0-9503-77910d6468d2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:10.888882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 364/735] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:10.889659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.913091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:10.913715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.914254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.914524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.924708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:10.926156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:10.939818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3cb3f2dd-d542-45f5-8c2b-f2761ab84c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:10.940613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:10.941556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:10.942484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:10.942484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:10.942484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:10.942484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 372/736] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:10.943064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445fb73-0eb3-441f-8b84-b0407bd01eaf None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.943279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445fb73-0eb3-441f-8b84-b0407bd01eaf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.943536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d445fb73-0eb3-441f-8b84-b0407bd01eaf None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.944741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 373/737] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:10.946823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e79f-af5b-4e10-a9c6-7ca567198016 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:10.947447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e79f-af5b-4e10-a9c6-7ca567198016 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:10.947927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3577e79f-af5b-4e10-a9c6-7ca567198016 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:10.949264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 374/738] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:10.949969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577e79f-af5b-4e10-a9c6-7ca567198016 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:10.969338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:10.996826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a28a73-faf9-460e-90d3-5a136e273133 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.030185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.030982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.031265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.031543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.038792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.040474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:11.051425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:11.052533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:11.053675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:11.054292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3577e79f-af5b-4e10-a9c6-7ca567198016 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.054377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 365/739] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:11.054377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.090528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.091209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.091495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.091775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.099248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.100658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-b483cc99-e888-4c5a-baf4-90cadc00f6c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:11.111919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:11.112823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:11.113785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 375/740] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:11.114641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708a20-33f4-42c1-8343-42bee865d271 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.114746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708a20-33f4-42c1-8343-42bee865d271 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.115144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74708a20-33f4-42c1-8343-42bee865d271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.133136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-74708a20-33f4-42c1-8343-42bee865d271 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:11.134310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-74708a20-33f4-42c1-8343-42bee865d271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:11.154542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769107a8-f8d3-4598-bc4f-f7f55389a528 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.165096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.189795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.190457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.190715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.190994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.191634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:11.191940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.193307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.199523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 376/741] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:11.200877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.201217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.201553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.206699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 366/742] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:11.207701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.222667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:11.223846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:11.225843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.290198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 377/743] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:01:11.291020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.313266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954662-a61a-47b1-883d-aceaed90679d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.346251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.346984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.347338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.347616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.348337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:11.348645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.350056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.366869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 367/744] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:11.367648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.392514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9b1d1-fd71-4e53-b912-b060de2257ba None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.393421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.427969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.428699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.428987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.429259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.429992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:11.430303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.431782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.439431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.450698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 378/745] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:11.451537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.463039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5201a9b3-b334-4bf1-918e-056e43bca630 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.498069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.498760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.499044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.499319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.507396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.508875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:11.521530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:11.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:11.523904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:11.526902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 368/746] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:11.527961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.619337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.631089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe508c5-3da8-42be-8f8a-98e292898a48 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.663614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.664462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a060a49-cca0-4968-a71d-9d4b4abf2dde None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.664549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.664727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.664994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.665625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.667047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.681783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 369/747] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:11 2022] POST /identity/v3/services => generated 240 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:11.682690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.722799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.724116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.724583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.725037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.725897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.727543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:11.746977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 379/748] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:11 2022] POST /identity/v3/services => generated 251 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:11.748276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:11.822929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae820fa5-90a2-4496-991d-cc664dc0c473 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.844021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.888144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.889070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.889355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.889624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.911997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:11.919195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b2702-8a91-4d7a-9842-ca3298e5f7c7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:11.967460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:11.968459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:11.969252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:11.969252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:11.983034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:11.984860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.065336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.067859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.089989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 370/749] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:12.095431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.143207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.145566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.206024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 380/750] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:12.206968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.213691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37344d-e57b-4866-a75e-2f5d1f162e28 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.250266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.250855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.251139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.251936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.252668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:12.252982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.255067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.278589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 371/751] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:12.279968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.342326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f365c-7deb-4c99-8a5a-4b7d201f08a8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.390414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.391216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.391323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8c842-c4e1-42ce-8e12-448d054f04d5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.391713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.392121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.393003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.407942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.409426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:12.424263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:12.425351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:12.426350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:12.426350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.426350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:12.426350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 381/752] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:12.426589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.433463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.444062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.445322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.445849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.446600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.463237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.464850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:12.485216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:12.486420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:12.487520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:12.487520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.487520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:12.489487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 372/753] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:12.490423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.565424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022d736-dc19-49dc-96ac-709812e4fba1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.603539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.604492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.604822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.605179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.616410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.617942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.628468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05769018-ed95-44c0-8abc-ad8506bc4aa1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:12.632341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:12.633481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:12.634796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:12.634796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:12.634796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:12.640354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 382/754] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:12.641296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.675208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.676458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.676918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.677346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.678390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:12.679742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.681265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.700087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 373/755] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:12.700087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.711365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.714918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.772198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc046ea-093b-4c76-a91b-a3946fbacde9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.807629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.808378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.808757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.809206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.809869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:12.810467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.811753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.817748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7c2d8-8b58-45e8-b8c4-19d6afac3e77 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.828237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 383/756] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:12.829079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.867388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.868209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.868630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.869034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.869839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:12.870298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.871929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.885354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 374/757] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:12.886683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:12.941776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61db8-1bed-4d0f-a813-42753224574f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:12.960972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.967122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:12.983035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:12.983780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:12.984055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:12.984328 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:12.992371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:12.994120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:12.997439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1e05f-0a94-4e40-9301-9f9a733547dd None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:13.005712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:13.006860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.008039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:13.011645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 384/758] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:13.012491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.037440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.038383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.038860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.039656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.046992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.048805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:13.060633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:13.061719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.062753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:13.065425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 375/759] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:13.066328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.117940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6724070-6947-4840-aa13-af517e62f9d8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.154730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.155366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.155642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.155917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.163436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.165375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.173538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637348be-b403-473c-9db9-c2e32cdf6493 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:13.176998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:13.177992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 385/760] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:13.179180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.210884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.211754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.212179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.212854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.213684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:13.214133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.215752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.217230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:13.218964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:13.234715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 376/761] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:13.234715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.298498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df6555-91e7-4c9c-abf3-6f0bbecb3840 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.335882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.336804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.337243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.337660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.338631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:13.339130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.340793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.342325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b99ff-e962-4699-8587-23e4443f7aa7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.355304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 386/762] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:13.356522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.378164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.378736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.379023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.379298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.379997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:13.380305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.381728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.394319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 377/763] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:13.395254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.451340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:13.464031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:13.471479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71f5f68-26f8-4d24-a8c5-3c9bdfafe67e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.502016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b1090-da97-49ed-a415-6711bd9f7857 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.509162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.510051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.510481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.510962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.511967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=70fc784da233409a9fa518088a4ac68d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:13.512427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.514115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.526680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 387/764] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/role_assignments?user.id=70fc784da233409a9fa518088a4ac68d&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:13.527947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:13.538912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.539791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.540073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.540351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.541090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=292584795f3c4e10b299942d80455a67) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:13.541404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.542881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fcefeb-b031-48be-98d6-66aa82adca18 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.554619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 378/765] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:13 2022] GET /identity/v3/role_assignments?user.id=292584795f3c4e10b299942d80455a67&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:13.715637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd35863-cf46-4b9a-b16f-28e26a12c85c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:13.756257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:13.756916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.757195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.757471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.759404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=1561e7c10ad04eeda2e31c3e31cf3c32, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:13.760829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651976af-5661-4f0b-b653-328ffbcc4c71 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:13.775259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 388/766] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:13 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:13.870136 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d5-1ead-4198-b010-c52405df0934 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.870676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d5-1ead-4198-b010-c52405df0934 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.870773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d5-1ead-4198-b010-c52405df0934 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.872349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 379/767] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:13.878043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.878317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.878610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.905876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:13.907111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:13.932055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:13.985613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180072-2e1b-42d4-8696-523a80feaecd None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.986461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180072-2e1b-42d4-8696-523a80feaecd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.986677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c180072-2e1b-42d4-8696-523a80feaecd None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:13.988231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 380/768] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:13.993722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:13.994042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:13.994285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.010435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 389/769] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:14.016268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:14.018022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:14.036815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.119783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 381/770] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:14.165345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.176957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26572a0-9ba0-4fcb-bad9-d5f438281e34 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.225967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:14.244311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.244977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.245262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.245535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.246127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.247604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:14.268238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:14.269304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:14.270162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 390/771] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:14.277447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.281937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b9fd0-c355-423b-8124-16685cb73fe9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.303803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:14.340741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.341390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.341675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.341951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] PATH_INFO: `/v3/services/network` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.342547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.347808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:14.367341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:14.368364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:14.369201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:14.375950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 382/772] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:14.388528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.466942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43202dd7-07bb-43ec-8154-d07f6b72359c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.542368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.542939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.543222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.543497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.544139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:14.544443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.545806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.559139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 391/773] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:14.563704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.585314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26436e76-b3e9-4f5e-ab8b-446973c05315 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.614955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:14.664956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:14.683611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544c57d-d099-49e7-80f7-48591ac6e590 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.697165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.697942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.698336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.698748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.699506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] RBAC: Adding query filter params (name=network) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:14.699940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.701420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.714798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 383/774] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:14.724008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.726563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.727195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.727479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.727752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.728391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:14.728691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.734470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.746946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 392/775] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:14.751762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.889417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f22047-7b6b-44e8-800c-66ab60c43e85 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:14.931917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:14.932562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:14.932856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:14.933134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:14.934923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] RBAC: Adding query filter params (interface=public, service_id=bf0952675c414f12946f4c5fe3a5a1f9, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:14.935252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:14.937366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:14.947391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 393/776] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/endpoints?service_id=bf0952675c414f12946f4c5fe3a5a1f9&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:14.953224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:14.971194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:15.036826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f75bf-4099-4b03-8206-b099e989d70d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:15.067929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76663349-0785-441c-89c9-f11d5479b1ea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:15.104248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:15.104861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.105152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.105434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:15.106053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:15.107523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:15.113725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 394/777] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services => generated 772 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:15.121424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:15.121985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.122269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.122546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:15.123224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] RBAC: Adding query filter params (type=network) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:15.123531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:15.124929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:15.139484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 384/778] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:15.145843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:15.270402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:15.306807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0ad45-0a54-4dcd-9caf-d50ec7bfa372 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:15.345713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:15.346330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.346661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.346945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:15.347663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] RBAC: Adding query filter params (interface=public, service_id=0932dfdc2c22447cbbef61c4697c9248, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:15.347975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:15.349416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac4825-c2fd-4c32-8bc9-6b853e153d75 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:15.355177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 395/779] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:15 2022] GET /identity/v3/endpoints?service_id=0932dfdc2c22447cbbef61c4697c9248&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:15.360603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:15.407376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:15.482749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f6228-ac3a-402e-a72a-2a5b3eca0344 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:15.527352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:15.527968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.528251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.528524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:15.529143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:15.530509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320c4d73-add5-4b83-b7ec-9020887907bf None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:15.538236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 385/780] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:15 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:15.973241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fa92-6586-4d9d-ba8c-14e3a340e747 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.973747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fa92-6586-4d9d-ba8c-14e3a340e747 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.973845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60fa92-6586-4d9d-ba8c-14e3a340e747 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:15.975417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 396/781] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:15.980906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:15.981203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:15.981486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.000530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:16.001753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:16.012182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ad10d-3cd5-4351-88cf-8cf8097f7336 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.012477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ad10d-3cd5-4351-88cf-8cf8097f7336 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.012742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ad10d-3cd5-4351-88cf-8cf8097f7336 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.014307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 397/782] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:16.019363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.019653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.019928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.044035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:16.046652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:16.112997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 386/783] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:16.135122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.169485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 398/784] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:16.171139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:16.191798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.271420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:16.299830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b11b9-1466-4b20-b970-8dfa502b7312 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:16.343672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35698b3-6404-43d2-8df4-e0e2abcc719f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:16.379342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:16.380051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.380339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.380613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.403778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-caa77719-4cf7-4d06-b0ea-3656b3da1c0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:16.409544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:16.410493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 387/785] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:16.417029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6a07-06df-432f-83b7-511ace1e3bba None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.417818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6a07-06df-432f-83b7-511ace1e3bba None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.417818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a6a07-06df-432f-83b7-511ace1e3bba None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.417941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:16.417941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.417941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.418151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.420439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 388/786] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:16.421141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6a07-06df-432f-83b7-511ace1e3bba None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.442511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ec909892-8777-4d95-a993-20cc7ac383b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:16.452003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:16.452960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:16.453829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:16.461413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 399/787] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:16.462795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.463090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.465692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.498543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:16.500253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:16.575184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:16.585323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a6a07-06df-432f-83b7-511ace1e3bba None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:16.614877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 400/788] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:16.616091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.622697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332cea-244d-4774-889b-0462283d3109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:16.623315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332cea-244d-4774-889b-0462283d3109 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.623598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332cea-244d-4774-889b-0462283d3109 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.623875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33332cea-244d-4774-889b-0462283d3109 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.624548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332cea-244d-4774-889b-0462283d3109 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:16.624852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332cea-244d-4774-889b-0462283d3109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:16.630619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33332cea-244d-4774-889b-0462283d3109 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:16.636167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 389/789] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:16.645264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332cea-244d-4774-889b-0462283d3109 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.714118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:16.756434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33332cea-244d-4774-889b-0462283d3109 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:16.776216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a645207-0271-43d6-8b04-3a0d4de52514 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:16.822345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:16.823963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:16.824569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.824850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.825118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.825789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:16.826092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:16.827366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:16.827656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:16.827931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:16.831514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:16.852722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 401/790] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:16.853693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.858210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:16.861957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:16.885798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:16.886889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:16.887777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:16.887777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:16.887777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:16.889606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 390/791] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:16.890661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:16.931164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:16.981804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aef8ea-c351-4d62-ab3f-d2d5f78801f8 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.001318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.021409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.022110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef701a3-3aa6-4239-b08e-dd6603aea453 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.022223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.022540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.022966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.034878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:17.040417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:17.041393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:17.042219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 402/792] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:17.042958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.071725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.072873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.073293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.074107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.087626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.090092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:17.113318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:17.114940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:17.116010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:17.116010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:17.116010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:17.118096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 391/793] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:17.119261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.159496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253740a4-bd71-48a4-8be4-c77b89c00fd0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.193714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.207572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.208459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.208943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.209384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.210197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.210706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.212651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.231464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 403/794] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:17.232622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.242822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67386e-5df4-4718-888f-db7dd6a474a7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.276274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.278437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.281619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.281619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.281619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.281619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.281619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.282568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.293539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 392/795] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:17.294478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.350457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bf91ba-7ab5-4014-a339-7e039038972d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.395376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.396782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.397495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.397910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.398968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.399414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.401323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.415463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a041f19-acfe-419c-a2cd-3c1cbd70e7cd None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.418631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 404/796] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:17.419587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.443764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.457643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.458283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.458558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.458867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.466405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.467896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:17.479810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:17.480788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 393/797] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:17.481727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.535706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.544549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cac24c-4517-44c4-b5be-ec83263e7fbe None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.579215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.579845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.580742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.581052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.591603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.593037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.597840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14024124-8572-41dc-8bf9-4bad9e98497f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7ff7317c-83b8-42c0-816c-976762112db8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:17.608283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:17.609264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:17.610245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:17.610245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:17.610245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:17.614718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 405/798] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:17.616021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a663-5a30-4f27-aa0d-fdc09c7b639f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.616382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a663-5a30-4f27-aa0d-fdc09c7b639f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.616663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551a663-5a30-4f27-aa0d-fdc09c7b639f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.619009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 406/799] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:17.619009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551a663-5a30-4f27-aa0d-fdc09c7b639f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.642460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.643615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.644153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.644429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.654658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.657191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:17.672429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:17.673381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 394/800] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:17.679254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.724354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.724763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551a663-5a30-4f27-aa0d-fdc09c7b639f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.760952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.761549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.761838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.762093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.762839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.763141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.764567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78439af-9077-49df-b205-cf65ae56fc79 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.782386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 407/801] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:17.783827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.784107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.784377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.789072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8d4ed-8b4b-4154-ba72-7d6a20cde420 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.797797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.805143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:17.806433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:17.825337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.825937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.826261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.826540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.827204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.827505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.828860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:17.845656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 395/802] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:17.846553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.890903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 408/803] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:17.892023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:17.955804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e589c-c67a-4c5f-ab1d-e12a0e619aaa None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:17.957643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:17.996569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:17.997481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:17.998029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:17.998302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:17.999067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:17.999376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:17.999651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca6162-46c8-431a-a7bc-c3dce4eea594 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.000856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.021209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 396/804] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.022013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.034790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.035417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.035696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.035969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.038661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.047917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.049900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:18.063771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.064760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:18.065683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:18.065683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:18.065683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:18.065683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 409/805] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.065683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.120263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d583e-8381-4a9b-818f-ce2d9b3cd964 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.152292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.152938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.153226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.153499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.161573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.162048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0084abd2-0024-4794-88bf-4032c0540c3d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.163751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-89433bc0-e17c-4aca-b0b8-742f9ce6c1cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:18.176049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.177021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:18.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 397/806] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.178804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa6f9-00ee-46f7-97b5-56fa9bea6f17 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.179087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa6f9-00ee-46f7-97b5-56fa9bea6f17 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.179361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffa6f9-00ee-46f7-97b5-56fa9bea6f17 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.180738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 398/807] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:18.181279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa6f9-00ee-46f7-97b5-56fa9bea6f17 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.198484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.199183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.199460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.199928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.200518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.201904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.208605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.210478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.216757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:18.217704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 410/808] 10.222.0.49 () {56 vars in 1129 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.218504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.267222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.287678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffa6f9-00ee-46f7-97b5-56fa9bea6f17 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.317758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe22d2a-d07c-4da4-9b35-716d0680daac None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.321970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.322615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.322891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.323160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.332887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.334332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:18.348792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.349716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:18.350645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:18.350645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:18.350645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:18.350645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 399/809] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.350876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.350976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.351050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.351050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.351620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.351942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:18.352239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.353605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489deb02-9a96-4e55-8946-e25ac8ddeb08 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.365086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 411/810] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.366230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.366508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.366806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.383166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:18.384331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:18.426945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.449157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 412/811] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:18.449974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.465467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fbda8c-f446-4b13-b3fd-034d28550c92 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.491931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.499142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.499746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.500027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.500301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.501026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:18.501330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.503368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.522532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 400/812] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.523392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.552605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee0b76-5b38-4f54-bcf8-1fa47ed0675d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.585435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.585981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.586260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.586534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.587186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:18.587492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.588847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.593115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 413/813] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.593868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.619667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291e592-fade-4d84-92d3-5e65a704232f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.646954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.653008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.653606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.653884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.654158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.654818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:18.655116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.656459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.668083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 401/814] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.668852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.695946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d575d-2871-4ad4-9a46-0f45e6c0e291 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.703032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.729919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.730530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.730835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.731105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.737458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.738908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:18.749441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.750458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 414/815] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.756961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.766500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe80fa5-6a7a-43da-9e3f-4d123f4c93ad None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.798823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.799975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.800257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.800524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] PATH_INFO: `/v3/services/network` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.801103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.802445 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:18.810694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:18.811853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:01:18.812712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:18.816849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 402/816] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/services/network => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:18.817505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:18.851638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92074364-acae-4b72-9a8e-42f9fa9efd3f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.874885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.884900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.885500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.885778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.886062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.918342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb0ae3a-b38c-4536-9c67-d0298ddcec4e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:18.919297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:18.952390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:18.952586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.952984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:18.953262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:18.953533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:18.953940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.954154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:18.954460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:18.956401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:18.960985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 403/817] 10.222.0.49 () {56 vars in 1124 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:18.961740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.050823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 415/818] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:19.051784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.061762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1a827-df5d-4610-bbec-a3695e109430 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.079425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.097220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.097829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.098109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.098374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.105105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.106491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.114850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:19.118387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:19.119464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:19.120357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 404/819] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:19.121248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.150774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6e534-345d-474f-9f6a-73c75c52c40d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.183482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.184076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.184358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.184636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.185319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:19.185622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.187080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.198665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 416/820] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:19.199423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.219018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3dbe7-5271-46cf-9bcd-0705af4f7df0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.251703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.252889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.253170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.253444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.254063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] RBAC: Adding query filter params (name=network) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:19.254364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.255766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.259982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 405/821] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:19.260733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.298967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27dc15-c95f-4977-8ad8-9a5313b261dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.304416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.324070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.333919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.334515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.334816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.335085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.335746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.337083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.357993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 417/822] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:19 2022] POST /identity/v3/endpoints => generated 346 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:19.358752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.362527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c54ab3-868b-4321-bfc4-a49454f5f13c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.406250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.406930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.407210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.407475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.420825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.422308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:19.437836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:19.439129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:19.440296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:19.440296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:19.440296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:19.443734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 406/823] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:19.444649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.487659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ae04-38c0-44cc-a38f-8c31e7d6fcb1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.533158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.534084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.534550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.535046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.535906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:19.536864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.537708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.549971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.551482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 418/824] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:19.552453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.561118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e47adb-ee3e-4bbd-b6c5-e3e2ccb5e91a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.601354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.601549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.602239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.602742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.603100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/70fc784da233409a9fa518088a4ac68d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.606250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=70fc784da233409a9fa518088a4ac68d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.608143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.621280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 407/825] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:19 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/70fc784da233409a9fa518088a4ac68d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:19.622211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.687302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0139cb-7a54-4693-901a-75a66f68b571 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.730016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878f5c3-87b6-4ed6-b5ba-098b4ecb9def None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.735310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.736180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.736654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.737102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.737908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] RBAC: Adding query filter params (type=network) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:19.738362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.740601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.748082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 419/826] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:19.749392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.768894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.769791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.770175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.770545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.771458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:19.771870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.773365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.787778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 408/827] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:19.788808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.811961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.875793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28dc2d1-9cc5-45f3-bff2-3ca5ea542c08 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.884482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:19.915904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.916623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.916910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.917191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/292584795f3c4e10b299942d80455a67/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.933146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876a898-a64c-43d4-8249-8c392344a1de None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:19.937431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=292584795f3c4e10b299942d80455a67, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.938909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:19.954369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 420/828] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:19 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/292584795f3c4e10b299942d80455a67/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:19.956002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:19.975370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:19.976314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:19.976787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:19.977451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:19.978419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:19.980052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:20.007158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 409/829] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:19 2022] POST /identity/v3/endpoints => generated 337 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:20.087172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6b2d2-3896-4b0f-8075-0de480510acf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:20.141906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:20.142490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:20.142813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:20.143140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:20.152556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:20.154056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:20.168017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:20.168965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:20.169891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 421/830] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:20.176279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:20.258902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:20.299531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84eb80-b7f0-49d1-982d-16c1e38aa0dc None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:20.337446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:20.338051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:20.338329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:20.338622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:20.339315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:20.339617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:20.341026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24372-dcf1-4298-835d-e50e43424ea1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:20.353789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 410/831] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:20.359362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:20.378435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:20.479474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0909862c-6b89-4b60-8ab5-1df7bedc75e5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:20.520512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:20.521134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:20.521420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:20.521692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:20.522437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:20.522775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:20.524194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83218b92-65cc-4c50-9381-ffb4e23f754a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:20.533059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 422/832] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:20 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:21.586487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155785cf-b96c-4cde-9ca6-664fb323d3c3 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:21.587070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155785cf-b96c-4cde-9ca6-664fb323d3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:21.587070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155785cf-b96c-4cde-9ca6-664fb323d3c3 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:21.588656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 411/833] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:21.593742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862da76-fc2a-4697-9108-ba01e7ec7ab3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:21.594124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862da76-fc2a-4697-9108-ba01e7ec7ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:21.594280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862da76-fc2a-4697-9108-ba01e7ec7ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:21.614212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9862da76-fc2a-4697-9108-ba01e7ec7ab3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:21.615524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9862da76-fc2a-4697-9108-ba01e7ec7ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:21.691113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 423/834] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:21.712786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155785cf-b96c-4cde-9ca6-664fb323d3c3 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:21.806045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19568e8-517e-4adb-8286-edeaf5690af5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:21.806373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19568e8-517e-4adb-8286-edeaf5690af5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:21.806615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19568e8-517e-4adb-8286-edeaf5690af5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:21.808209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 424/835] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:21.813297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:21.813580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:21.813859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:21.818146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155785cf-b96c-4cde-9ca6-664fb323d3c3 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:21.826698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:21.835220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:21.836802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:21.844659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:21.855796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:21.856717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:21.857122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:21.857581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:21.858312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:21.859949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:21.874893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:21.875909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:21.876741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:21.881882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 412/836] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:21.883304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:21.935538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 425/837] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:21.958371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.017996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d424e9-b231-4232-b98a-f2778b8b0962 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.063197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.064376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.065273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.065772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.066696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:22.067220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.069557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.081295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 413/838] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:22.081862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.088637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58035ec7-3f62-48bd-addb-4a0e98343ccb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.133591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.134650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.135126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.135648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.138703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.148776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:22.151986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e1fe9acf-034b-4f92-8b65-f659940ad51c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:22.153374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:22.154304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:22.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 426/839] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:22.156420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544122-3464-4a09-84bc-f0f2455975c3 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.156420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544122-3464-4a09-84bc-f0f2455975c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.156545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8544122-3464-4a09-84bc-f0f2455975c3 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.158434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 427/840] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:22.163649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544122-3464-4a09-84bc-f0f2455975c3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.206922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0cccf-cbba-4279-968a-e67c64f61c88 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.240654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.241281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.241562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.241834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.242492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:22.242812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.244196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36569f6-4cba-4f60-bb5f-32b0bed95d76 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.258604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 414/841] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:22.258604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.258604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.258604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.270513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:22.271830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:22.287138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8544122-3464-4a09-84bc-f0f2455975c3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.337563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.338789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.339472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.339892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.341307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:22.341889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.344207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.349231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 415/842] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:22.355157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 428/843] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:22.360728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.372448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.418261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.448544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.483736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9f8f7-e73c-44cd-80a9-ae7a552441b2 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.511838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b850bde-a11d-4cb0-9341-61371ebea10d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.521094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.521881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.522145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.522485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.533515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.535127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:22.551044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:22.552038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:22.552965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:22.552965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:22.552965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:22.556970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 416/844] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:22.557887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.559210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.560047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.560514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.561302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.576183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:22.584186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:22.585124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:01:22.585991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:22.593259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 429/845] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:22.594130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.681749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.684802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6adbe2-ff49-4e24-a7c5-503fa5912762 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.711210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede711c-e050-4a74-b59b-0414db3d2d9d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.721781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.722406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.722726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.723007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.723651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:22.723960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.725327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad3449-1794-49f6-880c-b46aee3c402c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.728214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.741947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 417/846] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:22.741947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1588-83e0-49b8-b7e6-f41aa400f497 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.741947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1588-83e0-49b8-b7e6-f41aa400f497 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.741947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de1588-83e0-49b8-b7e6-f41aa400f497 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.741947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 418/847] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:22.745319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de1588-83e0-49b8-b7e6-f41aa400f497 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.751588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.752195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.752474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.753404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.754079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:22.754388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.755784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9901759-8d3b-476e-a28c-f26da492828a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.765258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 430/848] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:22.766511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.766823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.767098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.786904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:22.788089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:22.856069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de1588-83e0-49b8-b7e6-f41aa400f497 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:22.872606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 431/849] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:22 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:22.873278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.895552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:22.896545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:22.897005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:22.897457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:22.904561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:22.905997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:22.917805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:22.918760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 419/850] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:22.925053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:22.942749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:22.994497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b57ad8-7c60-435c-8e86-60848d020042 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.006915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.025115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fc54a-3d2a-4aae-8789-e41b575f57d1 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.032800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.033424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.033703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.034026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.045297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.046764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.060070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.060701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.060983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.061259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:23.061374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:23.062346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:23.068177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:23.068177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:23.068177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:23.068384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.068468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 432/851] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:23.068559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.068640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:23.077299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:23.078451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:23.079600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:23.081440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 420/852] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 11:01:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:23.082190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.163907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.177347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659f9c9-478a-439a-ac2f-c4cb5176881f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.180672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435eba0-7e20-4c12-9707-d402a0a70bfb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.210751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.211342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.211618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.211883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.212601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:23.212902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.214285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.215456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.216152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.216572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.216873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.217580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:23.217948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.219649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.235968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 421/853] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:23.235968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.236186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 433/854] 10.222.0.49 () {56 vars in 1135 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:23.237787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.258378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.343890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7b8f-fe81-4da1-a3dc-310b31719799 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.344666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf9624-45a2-48b8-be8f-296cc6dec8f7 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.381501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.381855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.382355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.382465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.382771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.382915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.383027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.383352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.384068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:23.384461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.386592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.389986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.391626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 422/855] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:23.391747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.396793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:23.403199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:23.404327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:23.405481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:23.409611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 434/856] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:23.410361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.411852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.499358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.504866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0054385-fe42-41b1-bcc5-ab1e480a8d9d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.520450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e721de2-b894-4458-984f-c7faf2cddf9a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.537968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.538590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.538880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.539156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.568589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.569297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.569739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.570009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.570741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:23.571044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.572418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.586608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 435/857] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:23.586608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.626018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.627934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.641158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.707793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4cc49-8ad4-4e8e-8e17-f8638faa4834 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.745143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 423/858] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:23.750566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.752085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.752890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.753176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.753447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.754325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:23.754665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.756107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.777050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 436/859] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:23.782759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:23.787402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.885524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e5644-1ab7-4f2c-a82f-216a98af4ad9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.899223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:23.905470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63439d-65be-41a4-8ec7-ad1bd92e2a65 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:23.934168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.935513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.936234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.936655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.943608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:23.944215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:23.944497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:23.944771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:23.950680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:23.952309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:23.972681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:23.973850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:23.974996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:23.974996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:23.974996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:23.980619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 424/860] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:23.982400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.030458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.031850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.033875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.121237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f58f06-1f5b-4082-ac3a-aac70b3127a9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.151131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 437/861] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:24.152779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.165746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.167081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.167507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.167928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.168957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:24.169846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.171661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.193007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 425/862] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:24.193787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.224230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.275593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5630-7c98-4fca-af21-2c27929162d9 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.277932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.311323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b1982-089e-4743-8bd3-7da22a63cf86 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.324731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.326104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.326619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.327277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.341389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.343559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.347659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.348287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.348564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.348833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.356262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.357690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-474cdac4-c590-4f53-9b9b-5ecc0aad8f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:24.358823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:24.359717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:24.360612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:24.360612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:24.360612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:24.360612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 438/863] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:24.361405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379a42d-ae74-4e50-8d78-860ebb2780b4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.361684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379a42d-ae74-4e50-8d78-860ebb2780b4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.361957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379a42d-ae74-4e50-8d78-860ebb2780b4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.363697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 439/864] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:24.365368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379a42d-ae74-4e50-8d78-860ebb2780b4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-34763685-d2e3-41de-b3dd-f7db1a61c132 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:24.374336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:24.375292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:24.376594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 426/865] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:24.377345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.377628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.377947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.397343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:24.398526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:24.473543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 427/866] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:24.474189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.479338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379a42d-ae74-4e50-8d78-860ebb2780b4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.485422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.518997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.519737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.520021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.520294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.520934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:24.521241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.522616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.539679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 440/867] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:24.539679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.554195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.588947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353e3a5-3748-485c-aff7-9bf989f3c4b8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.625750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.626355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.626720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.626994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.627676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:24.628545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.629966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.641581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 428/868] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:24.642361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.648812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0ca7a-2a23-4173-b990-a868909677cf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.687168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.687780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.688186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.688463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.689112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.690515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.706861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 441/869] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:24 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:24.707625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.729394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.777044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949753f-b5b0-4a11-87be-6b66aa6d279c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.821694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:24.837081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.838116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.838398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.838701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.849658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.851249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.862680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287454e-5686-4203-9a5a-e57884376641 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:24.873525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:24.874422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:24.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 429/870] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:24.878910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:24.915147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:24.915914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:24.916492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:24.917079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:24.918559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=70fc784da233409a9fa518088a4ac68d) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:24.918914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:24.921359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:24.928608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 442/871] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/role_assignments?user.id=70fc784da233409a9fa518088a4ac68d&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:24.998905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23324684-c55c-4ec0-ac25-9afe6dff4329 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:25.028501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:25.038656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:25.039277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:25.039563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:25.039833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:25.040535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:25.040839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:25.042548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b16a8-525a-45e0-8edf-1abf7b5b4698 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:25.056690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 430/872] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:25.062254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:25.171116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4f0f-1fc0-400f-b567-65748cf0b078 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:25.205374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:25.205986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:25.206833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:25.207110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:25.207838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=292584795f3c4e10b299942d80455a67) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:25.208146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:25.209560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30abb3-b16a-4db4-a43b-d1facc217b26 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:25.215863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 443/873] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:25 2022] GET /identity/v3/role_assignments?user.id=292584795f3c4e10b299942d80455a67&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:25.630406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4705-933f-4094-9fa3-465b310af5ce None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:25.630829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4705-933f-4094-9fa3-465b310af5ce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:25.631008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f4705-933f-4094-9fa3-465b310af5ce None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:25.632524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 431/874] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:25.637830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdd1ff-ab87-4b54-9253-fa4150969be5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:25.638122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdd1ff-ab87-4b54-9253-fa4150969be5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:25.638395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afdd1ff-ab87-4b54-9253-fa4150969be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:25.644705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:25.658172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4afdd1ff-ab87-4b54-9253-fa4150969be5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:25.659910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4afdd1ff-ab87-4b54-9253-fa4150969be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:25.732174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 444/875] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:25 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:25.754866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f4705-933f-4094-9fa3-465b310af5ce None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:25.855452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f4705-933f-4094-9fa3-465b310af5ce None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:25.889428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:25.890267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:25.890747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:25.891190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:25.892057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:25.893588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dde1834-3f5d-4608-a68a-e7cb8cf7c460 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:25.914654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 432/876] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:25 2022] POST /identity/v3/services => generated 269 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:26.855872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fedb-068d-46d5-9516-1090638929b9 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:26.856382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fedb-068d-46d5-9516-1090638929b9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:26.856461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8158fedb-068d-46d5-9516-1090638929b9 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:26.858040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 445/877] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:26.863432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cd99-5c65-465f-a497-f27a0544e3fd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:26.864235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cd99-5c65-465f-a497-f27a0544e3fd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:26.864741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89cd99-5c65-465f-a497-f27a0544e3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:26.882801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3c89cd99-5c65-465f-a497-f27a0544e3fd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:26.883971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3c89cd99-5c65-465f-a497-f27a0544e3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:26.926797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:26.966689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 433/878] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:26.985321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158fedb-068d-46d5-9516-1090638929b9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.079123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1080e9-4ee9-4d9b-a48f-9108b01c8547 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.079457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1080e9-4ee9-4d9b-a48f-9108b01c8547 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.079684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1080e9-4ee9-4d9b-a48f-9108b01c8547 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.082628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 434/879] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:27.086789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.087081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.087358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.095152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8158fedb-068d-46d5-9516-1090638929b9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.107038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:27.108304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:27.127250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.133337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.134226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.134754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.135224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.136006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.137583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:27.146951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:27.147937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:27.148802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:27.151484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 446/880] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:27.158819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.194693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 435/881] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:27.212022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.263722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c7511-67da-454c-88f2-4062e1e4c44c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.298800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.299765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.300343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.300813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.301639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:27.302136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.303715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.308969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 447/882] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:27.316738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.326480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9b39d-2a8c-4920-a8c5-2a8d17836a43 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.328207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.361573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.362125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.362404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.363252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.363838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.365198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-9857f251-f771-478b-81e1-d96df35f7da2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:27.373732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:27.374798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:27.375652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 436/883] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:27.376474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778184c-6aa5-49c2-b13f-84a91f7d038b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.376551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778184c-6aa5-49c2-b13f-84a91f7d038b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.377224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3778184c-6aa5-49c2-b13f-84a91f7d038b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.378109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 437/884] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:27.382686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778184c-6aa5-49c2-b13f-84a91f7d038b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.395557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.441378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ff7ae-ab48-4f55-81ea-f3ce05ab9eae None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.477861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.478470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.478794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.479065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.479697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:27.479997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.481349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02611e5-f407-4854-8cb4-d6c4eef5e33a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.486152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 448/885] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:27.486660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3778184c-6aa5-49c2-b13f-84a91f7d038b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.487323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.488114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.488393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.507169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:27.508866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:27.526668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.527288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.527574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.527850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.528463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:27.528767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.530121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.535875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 438/886] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:27.536642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.569885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.591219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 449/887] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:27.592351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.649939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.659031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e848a5f-f0f8-4a3a-8758-238b5e9c24a4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.693779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362452-5d3f-4614-8fd8-41f1aaee02c3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.696129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.696807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.697148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.697502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.698264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] RBAC: Adding query filter params (interface=public, service_id=e381584329d642bc8fac86ebb6551c4e, region_id=RegionOne) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:27.698680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.700098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.708807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 439/888] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/endpoints?service_id=e381584329d642bc8fac86ebb6551c4e&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:27.709747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.733934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.734545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.734857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.735130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.735764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:27.736064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.737424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.750620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 450/889] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:27.750620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.843810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b534a5-f104-4e9f-ba44-d69bef484034 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.856640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ecedf-52f9-437e-bb5b-a67424e0dad1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:27.878824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.882042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:27.887432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.888263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.888612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.889141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.896831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:27.897445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.898364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.898685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.899311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:27.900739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:27.902749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-81009312-9985-466f-9c4d-47bd0aad331c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:27.907555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:27.908482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:27.909326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:27.910034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 451/890] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:01:27.910108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 440/891] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:27.919839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:27.974713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8205822-844a-4a33-8022-c6bf8afa183f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.975396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8205822-844a-4a33-8022-c6bf8afa183f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.975855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8205822-844a-4a33-8022-c6bf8afa183f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:27.977778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 441/892] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:27.983531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:27.984128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:27.984601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.003240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:28.005358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:28.036541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0d113e-2fae-418b-8ee5-990cb468a38a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.083631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.084243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.084518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.084783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.085455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.086322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.087759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.099104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 452/893] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:28.102074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 442/894] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:28.107618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.130176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.157479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.215944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c50a61-98da-4121-95e0-8d7423fc3573 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.248822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.264301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.265186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.265468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.265742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.269188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65dce2-c9dc-4630-8042-a7b4298d44c1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.279858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.281743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:28.302773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:28.303767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:28.304669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:28.304669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:28.304669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:28.304669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 453/895] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:28.308462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.309097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.309376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.309643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.310225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.311644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.315261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:28.321215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:28.322197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:28.323119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 443/896] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:28.328302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.437632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.439721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24832eb0-f877-46a0-a571-bf464d6a4156 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.445329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73c085-d95c-4e75-9464-94144fecd912 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.475685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.476309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.476570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.476833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.477463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.477762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.479269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.484947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.485812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.486268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.486766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.487589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.488109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.489250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 444/897] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:28.489862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.490493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.503977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 454/898] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:28.507074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.510065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.600492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc082c-3588-4ce7-8e77-2a7ab8df427d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.635402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.636104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.636387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.636659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.637504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.637731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.639126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.643891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4f1aa-0d62-47de-a016-fd72aaebffa6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.649501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 445/899] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:28.650846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.690280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.690957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.691254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.691657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.700107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.702181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:28.716550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:28.717542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:28.718474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 455/900] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:28.725191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.749088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.749839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:28.786755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0344b9e5-0d2c-4fd4-a54c-b4be6b9b92a5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.830968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.832472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.832977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.833643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.834881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] RBAC: Adding query filter params (interface=public, service_id=79c91e03fc2a438a8c24e2e73ea79bf4, region_id=RegionOne) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.835624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.837045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e728b-7056-433f-8cef-07778f8aec72 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:28.837503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.844450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 446/901] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/endpoints?service_id=79c91e03fc2a438a8c24e2e73ea79bf4&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:28.854245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.885516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:28.886136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:28.886395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:28.886695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:28.887537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:28.887850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:28.889237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:28.913543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 456/902] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:28.917153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:28.986603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366ca96-d67f-4bc2-a996-d7bd94d04567 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.021158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.029026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.029647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.029925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.030339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.031002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.033428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:29.042674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 447/903] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:01:29.063284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b6746-bc1a-4c0b-8004-148972fc2552 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.106217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.106877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.107225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.107501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.185938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.187670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:29.265392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.295105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 457/904] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:28 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:29.300509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:29.305367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.411370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fade605-f73c-4200-a1e3-8e8691aff72f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.447854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.448488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.448769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.449044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.460327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.461867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:29.476835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:29.477832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:29.478807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:29.478807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:29.478807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:29.478807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 448/905] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:29.484109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:29.600786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4101-6e75-4812-8207-f9fc48da3dcb None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.637084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.637746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.638030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.638358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.639037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:29.639345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.640761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183181c4-8fd6-4047-8d3e-b6318bbd3015 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:29.653410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 458/906] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:29.658654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:29.718692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.766141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7c0a-0957-4b22-88e8-8b4d967f3dc9 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.801036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.801670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.801951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.802227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.808981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.810420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:29.818894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790838-aa88-484d-b432-d56e9827c35a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.819180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790838-aa88-484d-b432-d56e9827c35a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.819457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75790838-aa88-484d-b432-d56e9827c35a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.821035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 459/907] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:29.822336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:29.824324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:29.826454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 449/908] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:29.827608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.827608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.827608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.830604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:29.844943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:29.846123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:29.874358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.925266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 460/909] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:29.947758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:29.953581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741ee5c-cf9c-4029-b68b-839beca3ce14 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:29.972003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:29.991752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:29.992381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:29.992634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:29.993188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:29.993691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:29.994001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:29.995453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e2b57-3b4e-4538-b11f-bce5bca743d0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.009980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 450/910] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:30.011723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9290f-c69c-4dc5-8f3c-91c392167795 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.012208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9290f-c69c-4dc5-8f3c-91c392167795 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.012622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9290f-c69c-4dc5-8f3c-91c392167795 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.014309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 451/911] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:30.016678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9290f-c69c-4dc5-8f3c-91c392167795 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:30.064742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bffa98-fd6c-4e33-ad7d-846112699490 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.102220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.102853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.103142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.103412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.104065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.105412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf18e97-f37a-4f39-bc6f-3994ea84502e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.120468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 461/912] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:29 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:01:30.121932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.122420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.122951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.143730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9290f-c69c-4dc5-8f3c-91c392167795 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.145110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:30.146654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:30.166660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:30.187574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.188204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.188485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.188761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.189524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] RBAC: Adding query filter params (role.id=09ccb2739ae84ef6a5caf06d441341cb, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=292584795f3c4e10b299942d80455a67) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:30.189829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.191906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.198647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 452/913] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:30 2022] GET /identity/v3/role_assignments?user.id=292584795f3c4e10b299942d80455a67&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=09ccb2739ae84ef6a5caf06d441341cb => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:30.243140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 462/914] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:30.267023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:30.312297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:30.388674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d7217-7ee4-4f1c-9e04-307fd161fd05 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.425903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.426542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.426850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.427122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.427712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.429144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:30.438041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:30.439132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:30.439970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 453/915] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:30.445171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:30.545739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402c840-4be3-49db-b41e-5ded5bdf2a2a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.580087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.580753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.581043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.581314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.582478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:30.582860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.584282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.589376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 463/916] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:30.594985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:30.701180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:30.705078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ffdca-8e93-484f-a12a-9a2602ef3bbd None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.742120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.743149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.743631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.744104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.745035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:30.745537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.747174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd0b93-9c33-41db-9cef-43f55f88802a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.754934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 454/917] 10.222.0.49 () {56 vars in 1114 bytes} [Wed Aug 10 11:01:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:30.762690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:30.785966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:30.870936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb27310-86dd-422f-a010-deacbb41ff42 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:30.906284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:30.906920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:30.907201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:30.907474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:30.908154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:30.909503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e4b3b0-60b6-4e14-bdab-4c6f320e4037 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:30.932463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 464/918] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:31.210989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df30158-b281-47c0-8c91-44edaacb4c9d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.211498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df30158-b281-47c0-8c91-44edaacb4c9d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.211575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df30158-b281-47c0-8c91-44edaacb4c9d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.213150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 455/919] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:31.218335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.218650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.218935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.236739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:31.237924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:31.308151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:31.313452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 465/920] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:31.340176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df30158-b281-47c0-8c91-44edaacb4c9d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:31.451888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df30158-b281-47c0-8c91-44edaacb4c9d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:31.488799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:31.489348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.489627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.489896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.490479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:31.491870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:31.500622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:31.501549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:31.502746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 456/921] 10.222.0.49 () {56 vars in 1132 bytes} [Wed Aug 10 11:01:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:31.510784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:31.618566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2bc4c-b5ed-47fc-8e98-d43a88fab987 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:31.653248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:31.653857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.654140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.654416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.655082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:31.655391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:31.656762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:31.662498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 466/922] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:31.668503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:31.699794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:31.777358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c81637-4588-4cf4-be1b-2903085fdaec None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:31.813016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:31.813965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.814480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.814987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.815817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:31.817161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:31.818770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66575f7-5427-4c83-a2f4-225fad5feb2d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:31.828860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 457/923] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:31.831699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:31.896764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:31.937677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82ba1c-2e22-4729-9c8f-1e64ea755962 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:31.983858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:31.984469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:31.984755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:31.985033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:31.985707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:31.987104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffd879-64f2-4c5b-bf29-b010dfca7834 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:32.011786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 467/924] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:31 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:32.244453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7f451-2509-47a4-9e3a-db1093d88f5f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.244974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7f451-2509-47a4-9e3a-db1093d88f5f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.244974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7f451-2509-47a4-9e3a-db1093d88f5f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.246542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 458/925] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:32.252716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c215-c06b-4b51-9ab9-0425bccdc660 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.252716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c215-c06b-4b51-9ab9-0425bccdc660 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.252716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3648c215-c06b-4b51-9ab9-0425bccdc660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.271558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3648c215-c06b-4b51-9ab9-0425bccdc660 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:32.272730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3648c215-c06b-4b51-9ab9-0425bccdc660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:32.351113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 468/926] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:32.372649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7f451-2509-47a4-9e3a-db1093d88f5f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.454383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5622-fb0d-4e52-8e61-5def7059984c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.454714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5622-fb0d-4e52-8e61-5def7059984c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.455026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588f5622-fb0d-4e52-8e61-5def7059984c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.456612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 469/927] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:32.462117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292086f-920d-4909-a4ad-dae763769910 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.462402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292086f-920d-4909-a4ad-dae763769910 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.462712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292086f-920d-4909-a4ad-dae763769910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.471227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:32.482132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7f451-2509-47a4-9e3a-db1093d88f5f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:32.485017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4292086f-920d-4909-a4ad-dae763769910 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:32.486422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4292086f-920d-4909-a4ad-dae763769910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:32.529737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:32.530296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.530589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.530860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.531443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:32.533040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:32.533795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:32.543958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:32.545243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:32.546135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 459/928] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:32.551571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.574614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 470/929] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:32.592011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292086f-920d-4909-a4ad-dae763769910 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.661447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa68d5-40bf-424b-a178-3d423ee49917 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:32.696354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:32.696965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.697239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.697510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.698142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:32.698445 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:32.698549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292086f-920d-4909-a4ad-dae763769910 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:32.699871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:32.705150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 460/930] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:32.711110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.735029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:32.735976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.736440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.736921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.747992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:32.752952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:32.754394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:32.755578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:32.762693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 471/931] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:32.763829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.774985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:32.801285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:32.825685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d99de-05cb-4c5b-b29b-9444954efe52 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:32.862805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:32.863804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.864216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.864654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.865417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:32.865915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:32.867711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:32.870240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18924e-d49b-427c-90cc-91a01caa4165 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:32.874104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 461/932] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:32.883973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.907302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:32.908049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:32.908434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:32.908813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:32.909637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:32.910041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:32.911739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:32.925351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 472/933] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:32.926418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:32.996548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.020230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590fc7f8-071a-4d07-b859-8744b1972bad None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.030035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.034296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83f276-47f6-4885-901d-20b44965be35 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.068043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.068575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.068909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.069123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.069823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] RBAC: Adding query filter params (interface=public, service_id=e499b1e00b114444bede06e156d7ebe3, region_id=RegionOne) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:33.070129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.071515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f7109-f5bc-4219-8e87-912bdc0803ff None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.071938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.072723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.073195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.073623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.076475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 462/934] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/endpoints?service_id=e499b1e00b114444bede06e156d7ebe3&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:33.082108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6da53-2b52-4317-a49d-bfb470347cc9 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.082413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6da53-2b52-4317-a49d-bfb470347cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.082718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e6da53-2b52-4317-a49d-bfb470347cc9 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.084082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 463/935] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:33.084626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6da53-2b52-4317-a49d-bfb470347cc9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.086335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.088085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-dd56ef08-1f36-40c3-9513-faf37e509d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:33.104506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:33.105495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:33.106403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:33.106403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:33.106403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:33.106403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 473/936] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:33.107058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.107323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.107580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.126287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:33.127508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:33.197220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 474/937] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:33.197841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e6da53-2b52-4317-a49d-bfb470347cc9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.205719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.239009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.239916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.240373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.240811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.241714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.243371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.250193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 464/938] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:01:33.251699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.272750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.323810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.354962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d80f3-829c-435b-b25a-adeab6b871d8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.384972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dc06b-8fd7-4909-8019-84b3f7d9135c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.407788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.408994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.409493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.409907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.411817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:33.412138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.414255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.429026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.429563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 475/939] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:33.429678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.429901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.430173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.430880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.432312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.434874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.434989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING keystone.server.flask.common [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 11:01:33.457229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.sql.core [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:01:33.457229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 11:01:33.457229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [parameters: {'id': '4d268930eef94157b71aa8d09e2043d4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 11:01:33.457229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80490) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d268930eef94157b71aa8d09e2043d4.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d268930eef94157b71aa8d09e2043d4. Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:01:33.479935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application result.read() Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 11:01:33.480814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application yield resource Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise self.value Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application session.commit() Aug 10 11:01:33.481811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:01:33.482771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:01:33.483675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application result.read() Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:01:33.484483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [parameters: {'id': '4d268930eef94157b71aa8d09e2043d4', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.487209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:33.490001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 4d268930eef94157b71aa8d09e2043d4. Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.492575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 465/940] 10.222.0.49 () {60 vars in 1140 bytes} [Wed Aug 10 11:01:33 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:33.494853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.565080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3fb9ed-5787-47c3-b395-c362c1cea1a3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.582006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.620449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.621447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.622017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.622522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.625952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363c0e9-4ab6-40fd-b420-d699fef596b3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.631204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.632914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.637437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:33.650635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:33.651706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:33.652648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:33.655212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 476/941] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:33.660945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.667129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.668081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.668358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.668629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.676707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.678355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:33.690452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:33.691559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:33.692557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:33.696561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 466/942] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:33.697476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.794413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8968b9-4c4d-4f42-8c98-1c4c1505dc09 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.809497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065ba80-d218-4c0d-a553-6afcd77d027d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:33.832965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.833848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.834285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.834739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.835610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:33.836061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.836729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.837835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.846795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:33.847410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:33.847699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:33.848599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:33.851876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:33.851876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:33.851876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:33.870546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 477/943] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:33.876589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 467/944] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:33.877836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:33.942021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:33.990130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498a01-14fd-4945-8efe-02d36a0bacd0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.033332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.034097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.034320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.034612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.047120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.049367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-eb132c9e-019c-4461-beff-fc88158bd76d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:34.066077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:34.067162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:34.068061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:34.068061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:34.068061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:34.068061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 478/945] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:34.072898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.173027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27635-de00-4431-83a6-43520ac2f2af None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.173322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27635-de00-4431-83a6-43520ac2f2af None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.173504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd27635-de00-4431-83a6-43520ac2f2af None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.175121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 479/946] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:34.180461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.180743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.181019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.181599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff41d2a-f6f0-4a03-bab5-e13e2eee5dd7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.199300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:34.200484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:34.216209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.216810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.217086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.217352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.218094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:34.218396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.219887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.232641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 468/947] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:34.238150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.270044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 480/948] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:34.291857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:34.292894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.337691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:34.366288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52d4a-34ee-45d9-8215-f0ea39327828 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.400796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4bb2d-6029-447c-b15c-7f2626840d8c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.408839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.409475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.409755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.410027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.418175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.419902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:34.433198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:34.434152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:34.435076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 469/949] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:34.440800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.442059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.442835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.443185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.443527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.444196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.445637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:34.454588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:34.456779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:34.459498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:34.462618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 481/950] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:34.463863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.554441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517164d-cedb-4567-8d51-922d71bcdf4a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.565328 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b672c6-94f1-456b-92bf-33f9fc53b98d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.567378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:34.580881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:34.601928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.602882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.603310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.603484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.603911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.604389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.604778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.605067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:34.605218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.605701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.606158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] RBAC: Adding query filter params (name=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:34.606550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.607595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.608303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.618400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 482/951] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:34.619675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.626687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 470/952] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:34.634525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:34.743326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c8bbc-661d-4d05-ab5c-c9a1900cacda None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.773438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec6245-1e4a-4d9c-99f6-1cc22126cfba None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:34.780436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.781132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.781478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.781819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.782521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:34.782950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.784447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0625c0-56a3-44c2-abeb-e6ef7f71ebb8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.794868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 483/953] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:34.816529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:34.818146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:34.818658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:34.819122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/292584795f3c4e10b299942d80455a67/roles/09ccb2739ae84ef6a5caf06d441341cb` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:34.821785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=292584795f3c4e10b299942d80455a67, role_id=09ccb2739ae84ef6a5caf06d441341cb)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:34.823500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e88880c-dc6e-4086-8a2b-f6ec16cca1bf None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:34.840545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 471/954] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:34 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/292584795f3c4e10b299942d80455a67/roles/09ccb2739ae84ef6a5caf06d441341cb => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:35.353691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c643d6-5067-4871-899a-6b2f5f76651f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:35.354208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c643d6-5067-4871-899a-6b2f5f76651f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:35.354208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c643d6-5067-4871-899a-6b2f5f76651f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:35.355823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 484/955] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:35.361538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:35.361914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:35.362089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:35.380438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:35.381926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:35.406195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:35.453736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 472/956] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:35.481486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c643d6-5067-4871-899a-6b2f5f76651f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:35.517990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:35.596223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c643d6-5067-4871-899a-6b2f5f76651f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:35.632862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:35.633835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:35.634318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:35.634833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:35.635679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:35.637238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:35.647152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:35.648380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:35.649353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:35.652417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 485/957] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:35.659675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:35.765486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041bafd-ff83-4576-a5ab-82e02946f19b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:35.800183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:35.800788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:35.801061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:35.801358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:35.801993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:35.802340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:35.803735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:35.812197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 473/958] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:35.814691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:35.923222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:35.929540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0f1eb-5128-4841-8f5b-2b8600de35dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:35.967330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:35.968324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:35.968878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:35.969341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:35.970171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:35.970701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:35.972256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7267bf3a-1968-4a91-a6ae-46dc73139c7d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:35.977884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 486/959] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:35.983413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:36.009949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:36.046136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a945a2-67e2-4295-8cda-a82eccecaa62 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.046675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a945a2-67e2-4295-8cda-a82eccecaa62 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.047180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a945a2-67e2-4295-8cda-a82eccecaa62 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.049332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 487/960] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:36.056959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.057366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.057738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.081432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:36.082899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:36.100103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91b124-2566-48bf-95ed-713b92de2bdb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:36.137228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:36.138182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.138665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.139084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.139877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:36.141484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:36.170969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 488/962] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:36.171324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 474/962] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:36.190105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:36.309048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:36.321628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f35b5-db0a-40e5-a0c8-c8a5bdf819c1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:36.364812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:36.365447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.365680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.365930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.366558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:36.367992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:36.370544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING keystone.server.flask.common [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 11:01:36.383788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.sql.core [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:01:36.383788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 11:01:36.383788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [parameters: {'user_id': 'cac94b64704947ebb99335f396762a3f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 11:01:36.383788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80491) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Aug 10 11:01:36.390471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:01:36.391481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application yield resource Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise self.value Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application session.commit() Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:01:36.392555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:01:36.393470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:01:36.394404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application result.read() Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:01:36.395980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cac94b64704947ebb99335f396762a3f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:36.396887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 11:01:36.397705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:36.398919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 489/963] 10.222.0.49 () {60 vars in 1131 bytes} [Wed Aug 10 11:01:36 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:36.403477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:36.488746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:36.514048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb73939-634d-4dbf-945e-d4bd412b16d7 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:36.549799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:36.550430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.550736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.551056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.559750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:36.561182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-d79b647e-cc4c-4587-8a3d-6756f2f461dd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:36.572941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:36.574014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:36.575256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 475/964] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:36.583609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:36.688146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46dced-bd03-40d4-ad0c-83cede30f7bc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:36.723650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:36.724626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.725105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.725595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.726511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:36.727040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:36.728854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c36e07-b5b2-4811-9d80-0823950e8b49 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:36.774780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 490/965] 10.222.0.49 () {56 vars in 1101 bytes} [Wed Aug 10 11:01:36 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:36.887520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b60d-4604-47dd-8d1c-7e1931c79fac None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.887885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b60d-4604-47dd-8d1c-7e1931c79fac None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.888168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1b60d-4604-47dd-8d1c-7e1931c79fac None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.889818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 476/966] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:36.896892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.897196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.897448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.912125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:36.922567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:36.924550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:36.986874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510044c5-ba0a-4990-89a7-6784ec98f9c0 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.987157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510044c5-ba0a-4990-89a7-6784ec98f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.987437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510044c5-ba0a-4990-89a7-6784ec98f9c0 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:36.988946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 477/967] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:36.994328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:36.995013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:36.995149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.002739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 491/968] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:37.017214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:37.018654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:37.026294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.102613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 478/969] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:37.124417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.126716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c557a-7631-4eee-8679-3dedc0f765d1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.164573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.165170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.165447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.165717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.166374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:37.167753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:37.172754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:37.185878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 492/970] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:37 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:01:37.210713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:37.231139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc982f30-7673-4730-8b1c-7b9dcf9bf524 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.275059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.275710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.275990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.276260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.287455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:37.292100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:37.293309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:37.299716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 479/971] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:37.300474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.411733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0cbb89-dfcc-4420-a2a7-196c9bea712b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.447707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.448650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.449126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.449580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.450442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:37.450959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:37.452478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:37.461795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 493/972] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:37.467546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.569925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f4a83-9cd0-4b87-a969-0d998eff4d8f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.593669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:37.604849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.605471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.605748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.606017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.616688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:37.618121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:37.632680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:37.633631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:37.634607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:37.634607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:37.634607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:37.635339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 480/973] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:37.639587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.701310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:37.744832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0f692-cff1-40a7-a7da-afc2a69f4b3b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.779454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.780067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.780348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.780621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.781265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:37.781571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:37.782963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:37.795373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 494/974] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:37.800726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:37.900074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14915f5b-0e70-4334-af02-8cf7a16d5fa0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:37.934001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:37.934660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:37.934942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:37.935213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:37.942755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:37.944187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:37.955054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:37.956006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:37.957049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 481/975] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:37.961840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.063586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38092eb2-f419-4165-99a8-23dcf48f6ebf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.080746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.099228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.099832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.100117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.100392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.101126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:38.101437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.102857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7738b8-1797-45e7-a139-82858396debb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.120481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 495/976] 10.222.0.49 () {56 vars in 1149 bytes} [Wed Aug 10 11:01:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:38.125804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.160089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.225925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1f21-1aaa-475c-adc8-f1c9fa3c88bf None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.258918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.259527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.259808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.260083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.324877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839424e-535f-4224-9056-287fdfc68c1c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.325212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839424e-535f-4224-9056-287fdfc68c1c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.325433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839424e-535f-4224-9056-287fdfc68c1c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.326939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 496/977] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:38.329455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.331040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.331697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.331987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.332266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.352388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:38.353621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:38.372787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.424440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 497/978] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:38.441820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 482/979] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:38.447386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.447527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.543738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b927c1-4e15-415a-b837-4593cf3f0045 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.548398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e446cb4-f863-4c8d-8b4f-8406e3766d47 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.551181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.580702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.581324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.581595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.581865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.586669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.587302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.587586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.587861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.588446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.589883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.592240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.592630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.594980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:38.599881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:38.601056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:38.602119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 483/980] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:38.607161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:38.614939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:38.615876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:38.616775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:38.616775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:38.616775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:38.617699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 498/981] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:38.622748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.718022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba43a2-80c4-4eb8-bd13-6e03db833208 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.733449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c4727-1ffc-40f5-906e-2c6b46d25e9e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.752942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.753556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.753837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.754113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.754789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:38.755097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.756495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.761391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 484/982] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:38.767109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.774908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.775865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.776292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.776708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.777613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:38.778087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.779744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.794065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 499/983] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:38.799703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.819961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.833426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:38.881733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa4b41-39c2-479d-b640-78b1422f67d5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.915447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dcec9-09e5-408d-8a82-8e8b3e134351 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:38.916398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.917007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.917290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.917562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.918725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:38.918850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.920124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.930223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 485/984] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:38.931246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:38.957051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:38.957934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:38.958219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:38.958612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:38.965407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:38.966902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:38.978894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:38.979843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:38.980735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 500/985] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:38.985495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.046507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195b07-d5b5-4f65-b93a-8bf50916978e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.067459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.073114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.083853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.084716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.084995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.085402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.086441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] RBAC: Adding query filter params (interface=admin, service_id=e499b1e00b114444bede06e156d7ebe3, region_id=RegionOne) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:39.086757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.088523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.091786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc158a6-b0c6-42f5-88ca-5b47810493a1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.099997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 486/986] 10.222.0.49 () {56 vars in 1249 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/endpoints?service_id=e499b1e00b114444bede06e156d7ebe3&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:39.101732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.127120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.127727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.128042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.128274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.128958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:39.129447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.130745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.143606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 501/987] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:39.150074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.215845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef256c09-12c2-4240-89da-f6bca7e2260f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.251081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa136e7b-cbc2-4744-8955-8382f7af3dd5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.252600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.253211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.253484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.253750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.254419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.255925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a07e5-fc77-4953-9cd3-a4a013df2b97 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.262085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 487/988] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:01:39.263313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56ce92-a709-4b1c-96e0-8da06ea0197b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.263600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56ce92-a709-4b1c-96e0-8da06ea0197b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.263872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f56ce92-a709-4b1c-96e0-8da06ea0197b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.265069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 488/989] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:39.271339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.271697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.271892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.288755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.289463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.289832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.290169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.291077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] RBAC: Adding query filter params (role.id=09ccb2739ae84ef6a5caf06d441341cb, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=292584795f3c4e10b299942d80455a67) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:39.291175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:39.291458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.292321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:39.293092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.310608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 502/990] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/role_assignments?user.id=292584795f3c4e10b299942d80455a67&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=09ccb2739ae84ef6a5caf06d441341cb => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:39.363258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.386764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 489/991] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:39.407655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.410778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.517680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf0b25-0875-4e0a-8aeb-b3ef30dcce6f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.551101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.551729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.552007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.552276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.552856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.554232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:39.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:39.563693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:39.569112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 503/992] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:39.573301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.678780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea4d77-10ef-44be-9515-fd78e1e7c684 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.713790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.714392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.714697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.714969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.715711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:39.716019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.717362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.726648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 490/993] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:39.727964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.799275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.834872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b58eb-f8df-45ff-9e0d-d0b0650ff947 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:39.876945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:39.877548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:39.877823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:39.878092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:39.878746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:39.879046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:39.880948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:39.890816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 504/994] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:39.891773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:39.942243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:39.992870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a919b9-2ab8-46e1-8a35-44ddd8fe399e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:40.026519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:40.027154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:40.027433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:40.027703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:40.028401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] RBAC: Adding query filter params (interface=public, service_id=2d8a73322483425aa7c39fbc5d5cebad, region_id=RegionOne) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:40.028708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:40.030059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd731d-dc95-4c25-9ebe-f45f77abaf7e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:40.039928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 491/995] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:39 2022] GET /identity/v3/endpoints?service_id=2d8a73322483425aa7c39fbc5d5cebad&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:40.040849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:40.139335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b832e-e1e3-4e6c-adf3-df2e0cc36a11 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:40.166460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:40.173389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:40.173983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:40.174261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:40.174528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:40.175212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:40.176581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb313799-f2d2-40be-addd-aea12a6de04f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:40.183330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 505/996] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:40 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:01:41.303619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4435a-ae49-4490-983f-7fd3bb979ec4 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.304159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4435a-ae49-4490-983f-7fd3bb979ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.304159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f4435a-ae49-4490-983f-7fd3bb979ec4 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.305520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 492/997] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:41.310637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.311022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.311192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.330372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:41.331578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:41.339094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd4be-a718-4225-94e0-21804c9a2881 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.339384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd4be-a718-4225-94e0-21804c9a2881 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.339665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cd4be-a718-4225-94e0-21804c9a2881 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.340999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 493/998] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:41.346042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.346325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.346627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.367617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:41.368785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:41.401371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 506/999] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:41.424217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.438372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 494/1000] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:41.460070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.504189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:41.530186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e3c81-f497-4632-9cc7-e32a6344f1c9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.554818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:41.564049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c083808-0ba5-4e47-ad86-09ad4e29079a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.566390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.567036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.567307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.567632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.568219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.569602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:41.578351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:41.582868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:41.584435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 507/1001] 10.222.0.49 () {56 vars in 1126 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:41.586420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.615168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.615806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.616099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.616363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] PATH_INFO: `/v3/services/image` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.616955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.618319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:41.627649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:41.628660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:41.634986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 495/1002] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:41.635813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.697578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfe957-45d0-4a2a-8c6f-020d96d5df6a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.731453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.732074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.732350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.732850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.733241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:41.733538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.734917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.739832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 508/1003] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:41.745290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.755877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c60ff-69b1-417c-aa15-079b3c0ba91b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.765232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:41.790626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.791222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.791501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.791771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.792402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:41.792698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.794305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.799768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 496/1004] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:41.805646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.811022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:41.860257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0e2216-1278-4878-bc9f-69805d439c64 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.897225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.897830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.898107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.898380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.899037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:41.899342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.900703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.905674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 509/1005] 10.222.0.49 () {56 vars in 1122 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:41.911109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:41.920463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b17d32-5105-4649-9ee7-f0785363f8f0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:41.957168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:41.957778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:41.958096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:41.958381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:41.959042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] RBAC: Adding query filter params (type=image) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:41.959353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:41.961008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f873a8-3f90-4595-8ac3-b1d7ec049c58 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:41.965470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 497/1006] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:42.022522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcdee-9999-4a00-8711-315ea493c733 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:42.043379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:42.067094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:42.068083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.068573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.069053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.069932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:42.071520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a0cb6-1347-43b5-8063-ac2444069b13 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:42.090229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 510/1007] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:01:42.209443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2805f5-61b7-4145-b25d-7c7cd3d6f9da None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.209810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2805f5-61b7-4145-b25d-7c7cd3d6f9da None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.210009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2805f5-61b7-4145-b25d-7c7cd3d6f9da None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.211618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 498/1008] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:42.217003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.217293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.217569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.236959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:42.238177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:42.320313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 511/1009] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:42 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:42.343446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2805f5-61b7-4145-b25d-7c7cd3d6f9da None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:42.382778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:42.452555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2805f5-61b7-4145-b25d-7c7cd3d6f9da None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:42.486388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:42.487052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.487335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.487608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.488188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:42.489574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:42.498079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:42.499235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:01:42.500076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:42.502654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 499/1010] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:42.505676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:42.609501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551db241-3324-4749-9ec9-b9581edeacb5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:42.610960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:42.646332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:42.646973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.647256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.647527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.648176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:42.648467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:42.649827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:42.659412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 512/1011] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:42.660646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:42.761492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:42.765144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c190b7e-3c4a-4250-b1f1-00ba3c7ae8f2 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:42.799899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:42.800508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.800789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.801064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.801693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:42.802028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:42.803417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea541205-65f1-4ff0-bebc-6662a9cf79e2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:42.810717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 500/1012] 10.222.0.49 () {56 vars in 1112 bytes} [Wed Aug 10 11:01:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:42.813867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:42.920102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22ad6-99fb-467f-990b-1eb537f3c18f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:42.954699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:42.955299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:42.955581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:42.955855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:42.956514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:42.957846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c283c13-8b4c-428c-b6f8-d602d2014fc8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:42.984176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 513/1013] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:43.963402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c90d-fe2e-4ecf-8cff-3ae81a34a118 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:43.963906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c90d-fe2e-4ecf-8cff-3ae81a34a118 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:43.964007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857c90d-fe2e-4ecf-8cff-3ae81a34a118 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:43.965412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 501/1014] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:43.970217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773589f-b4b1-4aff-b407-c214d19b777b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:43.970702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773589f-b4b1-4aff-b407-c214d19b777b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:43.970794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773589f-b4b1-4aff-b407-c214d19b777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:43.988483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6773589f-b4b1-4aff-b407-c214d19b777b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:43.989683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6773589f-b4b1-4aff-b407-c214d19b777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:44.006608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:44.064981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 514/1015] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:43 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:44.088051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857c90d-fe2e-4ecf-8cff-3ae81a34a118 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:44.141359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098d29a-c8cd-48e1-9d82-6947be4835f8 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:44.141570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098d29a-c8cd-48e1-9d82-6947be4835f8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:44.141845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098d29a-c8cd-48e1-9d82-6947be4835f8 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:44.143348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 515/1016] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:44.148581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:44.148870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:44.149147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:44.166697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:44.167884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:44.190193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857c90d-fe2e-4ecf-8cff-3ae81a34a118 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:44.227680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:44.228288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:44.228579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:44.228851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:44.229512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:44.230905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:44.243584 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 516/1017] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:44 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:44.250497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 502/1018] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:44 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:44.270178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:44.360944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:44.399713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e5c83-c879-4376-99d1-8c30304ecfa2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:44.440981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:44.441610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:44.441889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:44.442160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:44.443922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:44.445395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bd987e-3fdf-43fb-955d-968d4701626f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:44.447161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 517/1019] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:44.452920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:44.478783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:44.557208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9857fb-1c49-4d0c-b97e-65c317b9733a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:44.591949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:44.592566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:44.592850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:44.593123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:44.593760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:44.595280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51e2ba3-a440-4e5b-99d1-dc2c142f6a2b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:44.623068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 503/1020] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:01:44 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:45.145470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216a97-e01a-4784-9e4b-72c5030e015f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.145968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216a97-e01a-4784-9e4b-72c5030e015f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.145968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63216a97-e01a-4784-9e4b-72c5030e015f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.147387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 518/1021] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:45.152515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.152887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.153077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.171422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:45.172609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:45.238369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 504/1022] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:45.262218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63216a97-e01a-4784-9e4b-72c5030e015f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:45.369625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63216a97-e01a-4784-9e4b-72c5030e015f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:45.407654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:45.408193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.408467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.408791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.418916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:45.423991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:45.424951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:45.425831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 519/1023] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:45.431218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:45.440751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:45.534615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b29a43e-48d7-4d47-8c61-46be9239a986 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:45.569428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:45.570073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.570469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.570802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.571472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:45.571777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:45.573145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:45.581155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 505/1024] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:45.586773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:45.590106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:45.690125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d2231-54b8-4018-a489-eac3fb2d9edd None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:45.723624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:45.724766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.725046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.725317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.735950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:45.737391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:45.752915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:45.753890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:45.754877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:45.754877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:45.754877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:45.754877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 520/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:45.759822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:45.831307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:45.871104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5637-fda1-4bd2-8fc6-3834f600d5f3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:45.904080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:45.904684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:45.904966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:45.905237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:45.905878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:45.906179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:45.907587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:45.924260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 506/1026] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:45.925466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:45.981787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.026612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3871320-eca8-4e7a-9a7a-67ebd49312e4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.061935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.062636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.063045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.063418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.073429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.074885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:46.086131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:46.087106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:46.092895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 521/1027] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:46.093522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.192112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c355-f498-4d0e-86cf-ca0f8c2f700b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.219136 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.228395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.228928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.229211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.229483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.230275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:46.230596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.232060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c351a5e-e304-4ab0-8c59-042fc892cb5d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.253741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 507/1028] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:46.254964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.276395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103a53-20b0-4a0d-9bb7-eef6d232fd9f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.276684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103a53-20b0-4a0d-9bb7-eef6d232fd9f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.276966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17103a53-20b0-4a0d-9bb7-eef6d232fd9f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.278309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 508/1029] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:46.283047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.283334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.283613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.301470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:46.302703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:46.367102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ec984-c8e9-499c-8c0c-13869b139fe2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.385184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 509/1030] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:46.388136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.401335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.405847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.406596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.406965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.407239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.491929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.493272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.502767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfda17f-b557-473a-ad21-39e963f2e73c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.521498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.543248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.543884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.544162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.544686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] PATH_INFO: `/v3/services/image` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.545270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.546663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:46.556031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:46.557065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:46.558017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 510/1031] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:46.562903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.603131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 522/1032] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:46.608783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.627627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.674558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8f9ff9-d21c-49aa-85a4-82b52c45c936 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.712588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.713193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.713473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.713751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.714378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] RBAC: Adding query filter params (name=image) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:46.714718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.716075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532677ff-00d0-4c53-9cef-ebc00a784314 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.716406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d940e036-85f4-46f6-8982-99d12cc2d9a3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.720904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 511/1033] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:46.721878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3381-d3c0-482e-8a2c-b70ad9e9ed0b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.722162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3381-d3c0-482e-8a2c-b70ad9e9ed0b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.722438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f3381-d3c0-482e-8a2c-b70ad9e9ed0b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.723667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 512/1034] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:46.727402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f3381-d3c0-482e-8a2c-b70ad9e9ed0b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.753600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.754160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.754440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.754745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.764878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.766310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.773107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-4b57c61a-d3a9-4c3f-a6dc-5b326d841bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:46.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:46.781873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:46.782838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:46.782838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:46.782838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:46.786548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 523/1035] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:46.787672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.787958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.788234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.807264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:46.808474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:46.828432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f3381-d3c0-482e-8a2c-b70ad9e9ed0b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.862111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:46.862758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:46.863065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:46.863337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:46.863971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] RBAC: Adding query filter params (type=image) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:46.864325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:46.866674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:46.869846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:46.871289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 513/1036] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:46.872446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.882213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 524/1037] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:46 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:46.883276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:46.976289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601e0487-52de-4768-ba74-8fbda6947556 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.983833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f4a9f-95e5-4440-81c2-596f35127f24 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:46.984766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:47.010027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.010563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.010870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.011145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.011783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:47.012086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.013442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.018389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.019020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.019303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.019579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.020752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] RBAC: Adding query filter params (interface=public, service_id=fd940b829f624164a3df0663ab174345, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:47.021093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.022907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.025392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 514/1038] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:47.025644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.028272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 525/1039] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:01:46 2022] GET /identity/v3/endpoints?service_id=fd940b829f624164a3df0663ab174345&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:47.030471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.088174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:47.127275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b015-e497-4d68-a09c-b30e99b2649d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.128318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fa759-6f69-4178-bf45-890ff0ec9951 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.160376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.160996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.161279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.161395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.161554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.162561 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.162866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.163141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.163240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.164629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.166200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 515/1040] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:47.166970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.169883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.171338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:47.182870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:47.183920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:47.184964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 526/1041] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:47.185465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.208107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:47.272046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf528c3-eadc-4d90-9893-4459c248227c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.282190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b192ea8-103b-4b92-978f-5e1e48500849 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.302844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:47.305247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.305879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.306160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.306429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.307056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.308438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.314821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 516/1042] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:01:47 2022] GET /identity/v3/services => generated 1957 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:01:47.315710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.316767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.317372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.317652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.317933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.318625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.320664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.345023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 527/1043] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:01:47 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:47.435546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:47.440085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6852e-40a5-4773-9612-9fd94d9b558e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.480894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.481103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.481348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.481622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.482377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:47.482661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.484151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d3362-7bec-43f5-a481-b6a8a9bb2c71 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.498357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 517/1044] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:47.504375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:47.609689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b9e3b8-76e2-40e9-8173-afe56b3cff3a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:47.644124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:47.644729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:47.645041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:47.645276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:47.646011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] RBAC: Adding query filter params (role.id=09ccb2739ae84ef6a5caf06d441341cb, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:47.646917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:47.648391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edbf3e-d52c-45e3-95a6-ea8f3c30f77a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:47.654183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 528/1045] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:47 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=09ccb2739ae84ef6a5caf06d441341cb => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:49.346001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152b93-f604-43a0-85e0-7c318f4c0948 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.346538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152b93-f604-43a0-85e0-7c318f4c0948 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.346538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152b93-f604-43a0-85e0-7c318f4c0948 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.347962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 518/1046] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:49.352974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.353370 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.353529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.371392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:49.372590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:49.407626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:49.418818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a84e8-c461-433b-95ca-44718cb287a2 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.419381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a84e8-c461-433b-95ca-44718cb287a2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.419972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a84e8-c461-433b-95ca-44718cb287a2 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.421386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 519/1047] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:49.426364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.426678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.426957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.446703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:49.447873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:49.448455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 529/1048] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:49.470791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.514152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 520/1049] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:49.536800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.574957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f56c6a5-35ba-454b-a53f-d093b078c888 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:49.610882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:49.611443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.611726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.612000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] PATH_INFO: `/v3/services/image` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.612587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:49.613952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-56472928-7b4e-4fe2-a616-e465b5f7f764 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:49.623869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:49.624931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:49.625800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 530/1050] 10.222.0.49 () {56 vars in 1105 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:49.626651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a53f-11d4-4e44-b66a-758ec3ce9c1e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.626651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a53f-11d4-4e44-b66a-758ec3ce9c1e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.626799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe17a53f-11d4-4e44-b66a-758ec3ce9c1e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.630612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 531/1051] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:49.630612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17a53f-11d4-4e44-b66a-758ec3ce9c1e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.634716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:49.644682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b8da-a26a-4579-bc6e-303dd9b16ac8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:49.666926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:49.679045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:49.679677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.679953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.680224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.690010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6db4a392-488c-43be-81b4-7eec2025cc80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:49.694940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:49.695947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:49.696796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 521/1052] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:49.697487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.697908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.698184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.715386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:49.716559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:49.734657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe17a53f-11d4-4e44-b66a-758ec3ce9c1e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:49.766596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:49.767210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.767489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.767759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.768379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:49.768676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:49.770048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:49.776107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 532/1053] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:49.776866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.783587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 522/1054] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:49.784533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.869066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:49.883250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:49.886779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4d1e-175c-492a-b9bd-946fb6b025e3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:49.889425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd8f60-1f5e-424d-bd89-242591b26b16 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:49.919599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:49.920451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.920732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.921002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.921662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:49.921966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:49.923115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:49.923353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:49.923724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:49.924005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:49.924279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:49.924948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:49.925322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:49.926690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:49.931450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 523/1055] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:49.931909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 533/1056] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:49.934309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:49.937173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.029633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0b28-ac7e-46b9-be52-36e1533087f3 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.032807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b00482-8d4d-43ff-a1f2-3b8fa359d68f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.062736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.063349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.063626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.063893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.066143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.066777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.067057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.067324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.067985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.069860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.074658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:50.079389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:50.080825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:50.081884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 524/1057] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:50.082608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.087576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.094101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 534/1058] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:01:49 2022] POST /identity/v3/endpoints => generated 327 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:01:50.095107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.114713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.203177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec5c81-db75-4768-9e54-c2c19fbd7458 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.218245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369f27e-2037-4d2a-9c06-831ad88a576d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.241087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.241716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.241995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.242261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] PATH_INFO: `/v3/users/da1b12ee0093434996a11b6a3879e6ca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.258423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] RBAC: Authorizing `identity:get_user(user_id=da1b12ee0093434996a11b6a3879e6ca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.258764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.259322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.259614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.259890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.259991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.260564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:50.260870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.261547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 525/1059] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/users/da1b12ee0093434996a11b6a3879e6ca => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:50.262616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.267318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.272209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 535/1060] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:50.278527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.331867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.366751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.370463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d522bd98-e815-4b75-bba7-cf970b3a2e77 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.397038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e237d-9120-447b-b941-9d06a2c5a1c1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.408404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.409244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.409531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.409797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.435155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.436069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.436354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.436764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.448167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.449612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:50.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:50.465823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:50.466997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:50.466997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:50.466997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:50.470906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 536/1061] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:50.471844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.484309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.485940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.562050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.577376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4326279-5126-4aa3-8476-512b9228b0aa None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.604770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.613785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 526/1062] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:01:50.614936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.620164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.620778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.621059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.621333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.621983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:50.622287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.624117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.637113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 537/1063] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:50.642787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.720653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe6577-a7db-44dc-93b5-92e531d00168 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.747054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7a65ca-a198-437e-99b3-34d6b35c7581 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.755342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.756510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.756800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.757070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.758200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.759678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.766195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 527/1064] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:50.767249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.788763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.789394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.789675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.789956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.797194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.798657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.799275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:50.812021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:50.813084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:50.814042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 538/1065] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:50.819079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:50.871788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:50.882106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21700e90-4a57-4062-8d3b-b2597f165ead None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.916288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.916893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.917179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.917448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.918168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=7a32b02ea632453da2e7720513723e29, user.id=da1b12ee0093434996a11b6a3879e6ca) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:50.918477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.920175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:50.928731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389cf2a-6e38-403d-a38e-ecce0ca271e2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:50.930955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 528/1066] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/role_assignments?user.id=da1b12ee0093434996a11b6a3879e6ca&scope.project.id=7a32b02ea632453da2e7720513723e29&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:50.966889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:50.967662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:50.968006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:50.968359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:50.969220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:50.969597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:50.971424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.000999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 539/1067] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:51.002554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:51.103322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:51.111455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d595123-690d-4ab4-857f-582a31c72b2f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:51.144399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:51.145027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:51.145311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:51.145590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:51.156559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:51.158005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:51.172677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:51.173689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:51.175187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:51.175187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:51.175187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:51.175187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 529/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:51.179548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:51.280931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2498bc3-4da2-4f1a-b31c-ed0f631889e6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:51.283175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:51.316384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:51.316986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:51.317265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:51.317535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:51.318170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:51.318505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:51.319887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.336302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 540/1069] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:51.337936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:51.441598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d955d3-0c74-4365-af8e-e4ef7f6130e8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:51.477981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:51.478630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:51.478912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:51.479182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:51.485593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:51.487060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:51.497809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:51.498727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:51.504409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 530/1070] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:51.506820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:51.606807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7d7d2-468a-42b9-918c-8863fdb2038e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:51.639547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:51.640151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:51.640427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:51.640693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:51.641379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:51.641682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:51.643147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daba51c-1080-4aa8-ae2e-e9f2dd3e241e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.661013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 541/1071] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:51.662539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:51.672183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:51.766774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b4d74-df3c-454c-a1fe-3ce7f3ff6635 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:51.805300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:51.805950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:51.806232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:51.806503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/09ccb2739ae84ef6a5caf06d441341cb` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:51.808646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=1561e7c10ad04eeda2e31c3e31cf3c32, role_id=09ccb2739ae84ef6a5caf06d441341cb)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:51.810144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf67d68b-ee23-45d3-9e9a-1a8573d6c618 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:51.826427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 531/1072] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:51 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/1561e7c10ad04eeda2e31c3e31cf3c32/roles/09ccb2739ae84ef6a5caf06d441341cb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:52.142678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0c0d-0242-452c-b6d1-963aa5a00c05 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.143030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0c0d-0242-452c-b6d1-963aa5a00c05 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.143248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e0c0d-0242-452c-b6d1-963aa5a00c05 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.144670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 542/1073] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:52.149702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.149992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.150266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.169334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:52.170815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:52.232433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:52.242619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 532/1074] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:52.263799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e0c0d-0242-452c-b6d1-963aa5a00c05 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:52.289669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:52.367016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e0c0d-0242-452c-b6d1-963aa5a00c05 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:52.401404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:52.401932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.402215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.402491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.469262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:52.470723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:52.494741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:52.572813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 543/1075] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:52.578461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:52.675450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81681b-d6a2-4014-9697-e5a3c1fe86ac None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:52.708916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:52.709560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.709836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.710106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.719811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:52.721243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-7928508a-9c6f-4f70-802d-2cff030b93ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:52.735146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:52.736088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:52.737001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:52.737001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:52.737001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:52.737001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 533/1076] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:52.743689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:52.842245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07c44fd-06d7-4b18-9821-15e4198604d3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:52.872834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:52.886068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:52.886692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.886971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.887241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.887876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:52.888175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:52.889527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd08f6d-666d-4230-8947-1fa3c97888ff None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:52.906624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 544/1077] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:52.952402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6ba7-b31d-4d5e-a39f-cf9e06773f69 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.952690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6ba7-b31d-4d5e-a39f-cf9e06773f69 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.952949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f6ba7-b31d-4d5e-a39f-cf9e06773f69 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.954347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 534/1078] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:52.960336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:52.960648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:52.960898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:52.980143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:52.981365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:53.062880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 545/1079] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:53.076821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f6ba7-b31d-4d5e-a39f-cf9e06773f69 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:53.130405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:53.186811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f6ba7-b31d-4d5e-a39f-cf9e06773f69 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:53.220817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:53.221371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.221649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.221916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.232023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:53.236572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:53.237675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:53.238529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 535/1080] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:53.244411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:53.339165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdbeea-38e8-4bef-8b07-cae257c18aeb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:53.353151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:53.372767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:53.373364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.373643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.373914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.374599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:53.374906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:53.376287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:53.384060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 546/1081] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:53.389805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:53.486238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e697479-deb0-421b-a605-d469775e4681 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:53.495207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:53.519299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:53.519926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.520208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.520477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] PATH_INFO: `/v3/users/da1b12ee0093434996a11b6a3879e6ca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.521613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] RBAC: Authorizing `identity:get_user(user_id=da1b12ee0093434996a11b6a3879e6ca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:53.523613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:53.525401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 536/1082] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:01:53 2022] GET /identity/v3/users/da1b12ee0093434996a11b6a3879e6ca => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:53.530955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:53.628993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e91da8-0ffa-4a49-afa6-4d0bc052fb05 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:53.662475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:53.663116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.663394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.663664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.664759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:53.666197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8e80c-20ac-4543-90f9-c7d92a759421 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:53.667990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 547/1083] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:01:53 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:53.673952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:53.772561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8079b7e9-7189-43b4-8589-d21953a6888b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:53.805531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:53.806180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.806462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.806764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29/users/da1b12ee0093434996a11b6a3879e6ca/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.808633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] RBAC: Authorizing `identity:create_grant(project_id=7a32b02ea632453da2e7720513723e29, user_id=da1b12ee0093434996a11b6a3879e6ca, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:53.810062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7149c62-c189-4278-a2e7-034e4a95ccef None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:53.823262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 537/1084] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:01:53 2022] PUT /identity/v3/projects/7a32b02ea632453da2e7720513723e29/users/da1b12ee0093434996a11b6a3879e6ca/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:01:53.871229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440cb2d-fc7d-495e-9126-74e956b84fc5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.871498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440cb2d-fc7d-495e-9126-74e956b84fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.871771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4440cb2d-fc7d-495e-9126-74e956b84fc5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.873333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 548/1085] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:53.879067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:53.879357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:53.879635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:53.899812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:53.901047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:53.931320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:53.974542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 538/1086] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:53.998918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440cb2d-fc7d-495e-9126-74e956b84fc5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.046093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:54.100673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4440cb2d-fc7d-495e-9126-74e956b84fc5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.132534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.133160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.133443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.133716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.143940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:54.149085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:54.150264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:54.151248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 549/1087] 10.222.0.49 () {56 vars in 1120 bytes} [Wed Aug 10 11:01:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:54.156231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.254339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b307bb-a0d5-4d3f-8791-150f9818f569 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.288126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.288724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.289002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.289268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.289919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:54.290219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:54.291612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:54.299592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 539/1088] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:54.305279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.400998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4bf34-6515-4662-b800-28d5f7b7211a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.403254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:54.433600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.434219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.434497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.434797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.446360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:54.447823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:54.461957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:54.463019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:54.463931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:54.463931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:54.463931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:54.463931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 550/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:54.468699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.497317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:54.567149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf16cb1-4f0b-4bc9-8463-51087ef71c1e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.599976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.600569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.600845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.601112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.601742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:54.602039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:54.603414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:54.614531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 540/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:54.619922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.714883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d17e4a-d081-4399-aa29-423a57393fea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.746342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.746984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.747259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.747524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.754841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:54.756896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:01:54.767396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:54.768333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:01:54.769774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 551/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:54.774167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.860868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:54.874603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8fd46-5485-4793-8bc2-35cce8e07fab None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:54.907867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:54.908490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:54.908769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:54.909040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:54.909788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:54.910094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:54.911582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:54.930323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 541/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:54.935901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:54.954780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.034288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea9a89-abae-470b-af64-e2b382dd0f7f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.066718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.067309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.067591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.067862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.133087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.134440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.150419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.229419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 552/1093] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:55.234945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.329429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b6b54-67e3-47b7-9727-1d3ffa16ef6b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.362508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.363152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.363430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.363698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.373474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.374921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.381898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:01:55.390040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:55.391193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:01:55.392402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:01:55.392402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:01:55.392402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:55.392402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 542/1094] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:01:55.396720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.492270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c080d8-1a09-4596-a8a9-ef1296a4fb65 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.508741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.525839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.526436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.526742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.527013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.527646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:55.527945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.529289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.539995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 553/1095] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:55.545895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.641963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad60916-5179-4647-93e5-3567015141d8 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.674140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.674782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.675061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.675329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.682281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.683736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-101effc0-88eb-41f9-944d-9f055c63c13c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:01:55.694993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:55.695838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:55.696741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 543/1096] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:55.702230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.800949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abb27b-6fbd-4f4c-ac21-035fb796128d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.821794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899dfbe-0b21-4eed-9617-9c5f6c176453 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.822253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899dfbe-0b21-4eed-9617-9c5f6c176453 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.822466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4899dfbe-0b21-4eed-9617-9c5f6c176453 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.823887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 544/1097] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:01:55.828628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.828912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.829182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.834194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.835149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.835459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.835731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.836434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:55.836740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.838135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.848360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:55.849547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:55.855238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 554/1098] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:55.856110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.890351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.904698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:55.918614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 545/1099] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:01:55.941598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:55.953558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a35985-4f31-49c5-b5d5-a382a36099cf None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:55.986711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:55.987310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:55.987586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:55.987855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:55.988579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] RBAC: Adding query filter params (role.id=09ccb2739ae84ef6a5caf06d441341cb, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=1561e7c10ad04eeda2e31c3e31cf3c32) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:55.988884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:55.990290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:55.996068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 555/1100] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/role_assignments?user.id=1561e7c10ad04eeda2e31c3e31cf3c32&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=09ccb2739ae84ef6a5caf06d441341cb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:01:56.041309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1d8fa-7aec-4731-a9c4-a5674bccb469 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:56.077554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:56.078201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:56.078479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:56.078778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:56.090147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:01:56.094910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:01:56.095839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:01:56.096716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 546/1101] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:01:56.102978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:56.202380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:56.205912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1de93-193b-47a9-a080-e491174939a4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:56.240772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:56.241373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:56.241650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:56.241918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:56.242592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:56.242896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:56.244245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:56.252823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 556/1102] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:56.258946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:56.309966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:56.361545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8eca0-f4e6-4545-937b-569ba640430b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:56.400185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:56.400813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:56.401093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:56.401367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] PATH_INFO: `/v3/users/da1b12ee0093434996a11b6a3879e6ca` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:56.402500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] RBAC: Authorizing `identity:get_user(user_id=da1b12ee0093434996a11b6a3879e6ca)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:56.404019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:56.405604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 547/1103] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:01:56 2022] GET /identity/v3/users/da1b12ee0093434996a11b6a3879e6ca => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:56.411609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:56.515485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98cbf4-f47a-48e0-9b80-d15d2979f074 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:56.551653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:56.552262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:56.552542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:56.552812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:56.583580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:56.622967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:56.624408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:56.723421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 557/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:01:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:01:56.729977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:56.834305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d49017-3ec7-40d0-b213-5142fc3a9989 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:56.850402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:56.867887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:56.868505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:56.868783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:56.869050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:56.870135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:56.871570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e95e6a2-4bcf-4452-899c-fa024636707b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:56.873185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 548/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:01:56 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:56.879255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:56.938320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:56.975165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151739b4-1a8e-4684-a865-81ddfcbe9bf2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:57.007569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:57.008148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:57.008424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:57.008692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:57.009403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=7a32b02ea632453da2e7720513723e29, user.id=da1b12ee0093434996a11b6a3879e6ca) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:01:57.009705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:57.011200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2629b8-6d3b-4991-b7fa-357bfc547f32 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:57.016643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 558/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:01:56 2022] GET /identity/v3/role_assignments?user.id=da1b12ee0093434996a11b6a3879e6ca&scope.project.id=7a32b02ea632453da2e7720513723e29&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:59.000158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab96528-b0e2-4e5e-96d6-e0ce4fc29682 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:59.000671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab96528-b0e2-4e5e-96d6-e0ce4fc29682 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:59.000671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab96528-b0e2-4e5e-96d6-e0ce4fc29682 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:59.002016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 549/1107] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:01:59.007710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:59.008147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:59.008242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:59.026182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:01:59.027456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:01:59.091300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 559/1108] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:01:59.114786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab96528-b0e2-4e5e-96d6-e0ce4fc29682 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:59.212352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab96528-b0e2-4e5e-96d6-e0ce4fc29682 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:59.244124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:59.244734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:59.245015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:59.245291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:59.246391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:59.247834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbcaf-2a3c-495d-b152-54bd94b85cfd None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:59.249362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 550/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:01:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:01:59.254875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:01:59.307762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:01:59.349044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fd846c-626c-4c65-a7a5-95b2afa865a9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:01:59.381036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:01:59.381563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:01:59.382397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:01:59.382710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:01:59.383379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:01:59.384804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5bef6-c2d9-46f5-8fd2-6af8de214eed None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:01:59.404701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 560/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:01:59 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:01.426127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd773a8-984a-450d-926e-999773ced375 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:01.426690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd773a8-984a-450d-926e-999773ced375 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:01.426773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd773a8-984a-450d-926e-999773ced375 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:01.428085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 551/1111] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:01.433103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:01.433536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:01.433639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:01.451515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:01.452681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:01.515643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 561/1112] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:01.543498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd773a8-984a-450d-926e-999773ced375 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:01.595440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:01.641063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd773a8-984a-450d-926e-999773ced375 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:01.673073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:01.673684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:01.673961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:01.674230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] PATH_INFO: `/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:01.675628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] RBAC: Authorizing `identity:get_role(role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:01.677027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:01.678269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 552/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:01 2022] GET /identity/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:01.684749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:01.725006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:01.779290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e55108-1283-4725-a246-aa753ba2a348 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:01.811429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:01.812038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:01.812313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:01.812582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] PATH_INFO: `/v3/users/7bfdba9402f5400a88d744f0eb8ec103` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:01.825700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] RBAC: Authorizing `identity:get_user(user_id=7bfdba9402f5400a88d744f0eb8ec103)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:01.827220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:01.828827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 562/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:01 2022] GET /identity/v3/users/7bfdba9402f5400a88d744f0eb8ec103 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:01.834707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:01.928682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178715c1-e267-42d1-8263-9f2a54047d0a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:01.947329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:01.961053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:01.961645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:01.961920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:01.962186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:02.025763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:02.027223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:02.118034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 553/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:02.123899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:02.216303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:02.219435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f310c6-6ce5-43fb-a09e-32547c8b97b3 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:02.252818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:02.253426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:02.253703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:02.253972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:02.255104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:02.256512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9c5b7-5cf5-41f9-b86c-c4529a048b65 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:02.258702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 563/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:02 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:02.264707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:02.284678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:02.359564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcde8e-1ead-4326-a68c-e92768c2be14 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:02.392027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:02.392609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:02.392888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:02.393160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:02.393873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=7a32b02ea632453da2e7720513723e29, user.id=7bfdba9402f5400a88d744f0eb8ec103) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:02.394181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:02.395620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78498d42-f3eb-431a-9ee1-377c6d6df41b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:02.400821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 554/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:02 2022] GET /identity/v3/role_assignments?user.id=7bfdba9402f5400a88d744f0eb8ec103&scope.project.id=7a32b02ea632453da2e7720513723e29&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:04.373244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4c53-252a-4c83-92a6-1c101c5a6374 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:04.373762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4c53-252a-4c83-92a6-1c101c5a6374 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:04.373762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0a4c53-252a-4c83-92a6-1c101c5a6374 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:04.375143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 564/1118] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:04.380147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a08a4-f183-40f6-b273-70861426b844 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:04.380535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a08a4-f183-40f6-b273-70861426b844 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:04.380685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a08a4-f183-40f6-b273-70861426b844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:04.399047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-239a08a4-f183-40f6-b273-70861426b844 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:04.400217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-239a08a4-f183-40f6-b273-70861426b844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:04.464265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 555/1119] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:04.487817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a4c53-252a-4c83-92a6-1c101c5a6374 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:04.588078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0a4c53-252a-4c83-92a6-1c101c5a6374 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:04.619991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:04.620602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:04.620881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:04.621149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] PATH_INFO: `/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:04.622516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] RBAC: Authorizing `identity:get_role(role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:04.623953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:04.625211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 565/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:04 2022] GET /identity/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:04.631180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a08a4-f183-40f6-b273-70861426b844 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:04.646124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:04.725185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a08a4-f183-40f6-b273-70861426b844 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:04.756989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:04.758120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:04.758394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:04.758688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] PATH_INFO: `/v3/users/7bfdba9402f5400a88d744f0eb8ec103` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:04.759788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] RBAC: Authorizing `identity:get_user(user_id=7bfdba9402f5400a88d744f0eb8ec103)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:04.761190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:04.763022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 556/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:04 2022] GET /identity/v3/users/7bfdba9402f5400a88d744f0eb8ec103 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:04.768587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:04.801796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:04.865277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab792dd-3c7a-4f26-8073-162ae2bddaad None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:04.898342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:04.898923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:04.899200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:04.899466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:04.900558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:04.901967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde1d9b-f1cc-4987-b5eb-b5f532c1269e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:04.903586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 566/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:04 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:04.909530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:04.987179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:05.004751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9369e-6613-4dfe-8601-382a89f1b3ba None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:05.035483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731300d5-b985-4597-ae07-855e9357918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:05.036113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731300d5-b985-4597-ae07-855e9357918a None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:05.036389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731300d5-b985-4597-ae07-855e9357918a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:05.036655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731300d5-b985-4597-ae07-855e9357918a None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29/users/7bfdba9402f5400a88d744f0eb8ec103/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:05.039281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731300d5-b985-4597-ae07-855e9357918a None admin] RBAC: Authorizing `identity:create_grant(project_id=7a32b02ea632453da2e7720513723e29, user_id=7bfdba9402f5400a88d744f0eb8ec103, role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:05.040686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731300d5-b985-4597-ae07-855e9357918a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:05.052717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 557/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:02:04 2022] PUT /identity/v3/projects/7a32b02ea632453da2e7720513723e29/users/7bfdba9402f5400a88d744f0eb8ec103/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:02:06.998282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80afd-67a4-461b-b98c-264134c37e99 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:06.998884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80afd-67a4-461b-b98c-264134c37e99 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:06.998884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80afd-67a4-461b-b98c-264134c37e99 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.000171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 567/1124] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:07.005164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.005580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.005722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.023582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:07.024751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:07.088242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 558/1125] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:02:07.111827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80afd-67a4-461b-b98c-264134c37e99 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:07.214317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80afd-67a4-461b-b98c-264134c37e99 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:07.216553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:07.246849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:07.247464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.247742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.248012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] PATH_INFO: `/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.249368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] RBAC: Authorizing `identity:get_role(role_id=63e54ac1a86e40ef97eb4ac1c6eebb21)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:07.250742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:07.252048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 568/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:07 2022] GET /identity/v3/roles/63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:07.258496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:07.324802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:07.354473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fec5f-02f3-4760-9dc5-30792a8d0982 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:07.387365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:07.387983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.388276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.388549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] PATH_INFO: `/v3/users/7bfdba9402f5400a88d744f0eb8ec103` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.389664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] RBAC: Authorizing `identity:get_user(user_id=7bfdba9402f5400a88d744f0eb8ec103)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:07.391117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:07.392671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 559/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:07 2022] GET /identity/v3/users/7bfdba9402f5400a88d744f0eb8ec103 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:07.399062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:07.492978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105e6e4-c508-41b3-a78e-fd3432638056 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:07.524819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:07.525397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.525674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.525943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.564062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:07.592291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:07.593703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:07.685635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 569/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:07.691432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:07.785432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62d62-c37d-430c-b5b0-39e33d198e21 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:07.818046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:07.818678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.818956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.819224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] PATH_INFO: `/v3/projects/7a32b02ea632453da2e7720513723e29` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.820917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] RBAC: Authorizing `identity:get_project(project_id=7a32b02ea632453da2e7720513723e29)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:07.822319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b26daff-3217-4734-be16-356349c218e6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:07.823937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 560/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:07 2022] GET /identity/v3/projects/7a32b02ea632453da2e7720513723e29 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:07.829872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:07.896563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:07.924586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b1d08b-2040-475d-bb8a-daebb2c7884a None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:07.956824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:07.957420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:07.957703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:07.957978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:07.958738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] RBAC: Adding query filter params (role.id=63e54ac1a86e40ef97eb4ac1c6eebb21, scope.project.id=7a32b02ea632453da2e7720513723e29, user.id=7bfdba9402f5400a88d744f0eb8ec103) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:07.959052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:07.960459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252ebddc-d4c6-4f41-a9c3-0b7363ce4b9a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:07.966011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 570/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:07 2022] GET /identity/v3/role_assignments?user.id=7bfdba9402f5400a88d744f0eb8ec103&scope.project.id=7a32b02ea632453da2e7720513723e29&role.id=63e54ac1a86e40ef97eb4ac1c6eebb21 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:09.961596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658e0b-8ab1-4645-9090-d5b1430c897a None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:09.962105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658e0b-8ab1-4645-9090-d5b1430c897a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:09.962212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d658e0b-8ab1-4645-9090-d5b1430c897a None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:09.963500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 561/1131] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:09.968719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:09.969117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:09.969261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:09.986624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:09.987789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:10.051350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 571/1132] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:10.074817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d658e0b-8ab1-4645-9090-d5b1430c897a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:10.085323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:10.170821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d658e0b-8ab1-4645-9090-d5b1430c897a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:10.202780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:10.203380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:10.203658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:10.203928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:10.205020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:10.206428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448c8c7-1e85-4cd7-86ae-8b6fba6eaba2 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:10.207973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 562/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:10.213662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:10.274844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:10.308377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0465d904-91ff-4b26-b0ba-86b96bb9bf91 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:10.341008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:10.341588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:10.341865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:10.342133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:10.342789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:10.344625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc8244-4d60-43ae-99c7-53980c8522fd None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:10.369706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 572/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:02:10 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:12.361929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901ecad-dccc-4eff-b7df-2da60d244582 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:12.362482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901ecad-dccc-4eff-b7df-2da60d244582 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:12.362482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901ecad-dccc-4eff-b7df-2da60d244582 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:12.363820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 563/1135] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:12.369449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:12.369830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:12.369999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:12.387510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:12.388682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:12.451890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 573/1136] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:12.475275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901ecad-dccc-4eff-b7df-2da60d244582 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:12.548751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:12.571929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901ecad-dccc-4eff-b7df-2da60d244582 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:12.603913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:12.604517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:12.604799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:12.605068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:12.606151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:12.607612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d671bac-d59a-4895-a17a-be62ff630bc3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:12.609132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 564/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:12.614883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:12.661175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:12.709530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf393fec-c096-4de9-b0b2-3335223fe9bd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:12.741234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:12.741811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:12.742087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:12.742357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:12.743628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:12.745060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520058ed-04c8-4bcb-b6bf-5a33b8ed970d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:12.764468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 574/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:02:12 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:14.774961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb8bd-6363-4f0b-9d40-92eec6937f54 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:14.775512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb8bd-6363-4f0b-9d40-92eec6937f54 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:14.775512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb8bd-6363-4f0b-9d40-92eec6937f54 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:14.776820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 565/1139] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:14.781962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:14.782346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:14.782514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:14.800750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:14.801912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:14.865849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 575/1140] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:14.889022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb8bd-6363-4f0b-9d40-92eec6937f54 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:14.984934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb8bd-6363-4f0b-9d40-92eec6937f54 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.016958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.017564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.017842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.018113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.028170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:15.032653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:15.033796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:15.034756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 566/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:15.039564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:15.071799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:15.135765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ad748-070c-4204-96b5-a49e77d4db0a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.168277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.168861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.169142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.169415 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.170071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:15.170373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:15.171758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:15.179904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 576/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:15.185515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:15.189836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:15.287217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71cca8c-e444-4d50-9f17-b2ae3c5f307b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.320054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.320668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.320945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.321214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] PATH_INFO: `/v3/users/4a0b558e9416480d930b2b4c99637903` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.334199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0b558e9416480d930b2b4c99637903)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:15.335664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:15.337718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 567/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:15 2022] GET /identity/v3/users/4a0b558e9416480d930b2b4c99637903 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:15.343530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:15.432563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:15.438252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b489a4-92f7-4607-979f-0bbc876698e6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.470819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.471409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.471688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.471960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.535351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:15.536772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:15.635956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:15.642509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 577/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:15.648552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:15.707694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:15.745529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918d0ae-b4c5-4ff1-8980-7f8a833b4dd0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.778796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.779405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.779721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.779998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] PATH_INFO: `/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.781529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] RBAC: Authorizing `identity:get_project(project_id=6f45a1be5b054542b384cfae7d9fbf1c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:15.783373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0509a0-57d1-47a7-85b6-884a1625e2db None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:15.784991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 568/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:15 2022] GET /identity/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:15.791430 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:15.887758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb65b5e-392e-41d2-ab64-1689fe92b869 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:15.919701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:15.920302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:15.920585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:15.920860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:15.921581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=6f45a1be5b054542b384cfae7d9fbf1c, user.id=4a0b558e9416480d930b2b4c99637903) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:15.921891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:15.923584 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ca1a9-d2ad-484a-813a-df17a61781ef None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:15.928737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 578/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:15 2022] GET /identity/v3/role_assignments?user.id=4a0b558e9416480d930b2b4c99637903&scope.project.id=6f45a1be5b054542b384cfae7d9fbf1c&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:17.867439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8e89-1115-4324-8ba8-7f289aef49ec None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:17.867881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8e89-1115-4324-8ba8-7f289aef49ec None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:17.867987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a8e89-1115-4324-8ba8-7f289aef49ec None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:17.869321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 569/1147] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:17.874851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:17.875211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:17.875398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:17.892716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:17.893875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:17.946824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:17.957687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 579/1148] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:17.980163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a8e89-1115-4324-8ba8-7f289aef49ec None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:18.076374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a8e89-1115-4324-8ba8-7f289aef49ec None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:18.103599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:18.109352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:18.109964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:18.110245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:18.110517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:18.120229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:18.124804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:18.125882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:18.127104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 570/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:18.131616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:18.224148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeae8e-d8fa-467c-83c7-93e85a4173d0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:18.257355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:18.257956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:18.258237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:18.258511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:18.259219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:18.259522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:18.260882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:18.268708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 580/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:18.274291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:18.372220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d29cd6-69c1-4a9d-a383-d6b000d9ef62 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:18.404610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:18.405211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:18.405491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:18.405763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] PATH_INFO: `/v3/users/4a0b558e9416480d930b2b4c99637903` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:18.406902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] RBAC: Authorizing `identity:get_user(user_id=4a0b558e9416480d930b2b4c99637903)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:18.408318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:18.409841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 571/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:18 2022] GET /identity/v3/users/4a0b558e9416480d930b2b4c99637903 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:18.415229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:18.465816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:18.511019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061556de-9733-42fc-a9c0-b14bf8c53432 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:18.542495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:18.543130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:18.543408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:18.543680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] PATH_INFO: `/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:18.544766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] RBAC: Authorizing `identity:get_project(project_id=6f45a1be5b054542b384cfae7d9fbf1c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:18.546172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaa79d7-5ecc-4982-8d3e-cea0440c687e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:18.547870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 581/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:18 2022] GET /identity/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:18.553945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:18.609824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:18.649523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208dbc8-e6cb-4794-b6fa-3e3b7a64ef82 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:18.681229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07829632-be86-4876-933a-e5a6f722288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:18.681858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829632-be86-4876-933a-e5a6f722288c None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:18.682135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829632-be86-4876-933a-e5a6f722288c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:18.682407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07829632-be86-4876-933a-e5a6f722288c None admin] PATH_INFO: `/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c/users/4a0b558e9416480d930b2b4c99637903/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:18.684303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829632-be86-4876-933a-e5a6f722288c None admin] RBAC: Authorizing `identity:create_grant(project_id=6f45a1be5b054542b384cfae7d9fbf1c, user_id=4a0b558e9416480d930b2b4c99637903, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:18.685715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07829632-be86-4876-933a-e5a6f722288c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:18.698367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 572/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:02:18 2022] PUT /identity/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c/users/4a0b558e9416480d930b2b4c99637903/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:02:20.654135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc75c0-d8c3-445d-b228-dd45c871ba1a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:20.654702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc75c0-d8c3-445d-b228-dd45c871ba1a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:20.654702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc75c0-d8c3-445d-b228-dd45c871ba1a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:20.656012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 582/1154] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:20.660975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:20.661336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:20.661530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:20.679204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:20.680374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:20.743640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 573/1155] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:20.766247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc75c0-d8c3-445d-b228-dd45c871ba1a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:20.863870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc75c0-d8c3-445d-b228-dd45c871ba1a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:20.882456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:20.896679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:20.897284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:20.897558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:20.897847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:20.908639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:20.913287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:20.914198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:02:20.915153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 583/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:20.920418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:20.968337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:21.017360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b333c-5931-4f76-92b1-7cf331d6acdc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:21.049543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:21.050130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:21.050412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:21.050723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:21.051374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:21.051675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:21.053045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:21.061236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 574/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:21.066792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:21.160933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98987a1d-4d7e-4006-ae05-710d19e81cea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:21.192557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:21.193157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:21.193432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:21.193702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] PATH_INFO: `/v3/users/4a0b558e9416480d930b2b4c99637903` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:21.194829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] RBAC: Authorizing `identity:get_user(user_id=4a0b558e9416480d930b2b4c99637903)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:21.196229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:21.198318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 584/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:21 2022] GET /identity/v3/users/4a0b558e9416480d930b2b4c99637903 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:21.203806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:21.295997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05050b-e990-464d-b5aa-8868c7e88006 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:21.305787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:21.329130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:21.329730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:21.330007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:21.330276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:21.393856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:21.395334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:21.488559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 575/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:21.494397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:21.541083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:21.593582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebefc-d054-4f00-b15b-3d435a13f11c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:21.625193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:21.625804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:21.626081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:21.626351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] PATH_INFO: `/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:21.627475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] RBAC: Authorizing `identity:get_project(project_id=6f45a1be5b054542b384cfae7d9fbf1c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:21.628882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517ddb57-7a9d-4cae-8bc5-65ec4a0aa2ba None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:21.630473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 585/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:21 2022] GET /identity/v3/projects/6f45a1be5b054542b384cfae7d9fbf1c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:21.636950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:21.647959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:21.731091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e36e5-53e2-4db9-b5ab-7de013fccf15 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:21.762891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:21.763418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:21.763695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:21.763965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:21.764673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=6f45a1be5b054542b384cfae7d9fbf1c, user.id=4a0b558e9416480d930b2b4c99637903) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:21.764979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:21.766377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee501e7-89fc-4485-8ebd-e88ba2b876a1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:21.771829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 576/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:21 2022] GET /identity/v3/role_assignments?user.id=4a0b558e9416480d930b2b4c99637903&scope.project.id=6f45a1be5b054542b384cfae7d9fbf1c&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:23.754336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084ef91-5045-4bae-87ad-48ace1058877 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:23.754898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084ef91-5045-4bae-87ad-48ace1058877 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:23.754898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084ef91-5045-4bae-87ad-48ace1058877 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:23.756236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 586/1162] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:23.761259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:23.761655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:23.761813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:23.779189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:23.780360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:23.851132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 577/1163] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:23.877747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084ef91-5045-4bae-87ad-48ace1058877 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:23.976106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b084ef91-5045-4bae-87ad-48ace1058877 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:23.991668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:24.008894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:24.009427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:24.009729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:24.009997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:24.074072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:24.075543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:24.168808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 587/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:24.174938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:24.177927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:24.268972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae0563-9485-472f-82b1-9a55909c9e9f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:24.303884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:24.304505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:24.304784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:24.305056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:24.311515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:24.312949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3d5ac9db-931d-4733-8409-8536674ffce3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:24.325451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:24.326636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:24.327656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 578/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:02:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:24.332475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:24.349635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:24.428044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d6aff-23d6-4e65-8aad-4eccd321a44c None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:24.460144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:24.460800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:24.461080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:24.461347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:24.461976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:24.462274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:24.464355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63f939d-1f17-4641-97d0-41d5a8b0b2e1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:24.471093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 588/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Wed Aug 10 11:02:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:26.433552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de21e-41fa-477a-946e-9fd63d66e757 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:26.434075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de21e-41fa-477a-946e-9fd63d66e757 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:26.434075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de21e-41fa-477a-946e-9fd63d66e757 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:26.435449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 579/1167] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:26.440482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed5931-dd38-4440-86ba-ec27851724c6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:26.440923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed5931-dd38-4440-86ba-ec27851724c6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:26.441037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed5931-dd38-4440-86ba-ec27851724c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:26.459131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-afed5931-dd38-4440-86ba-ec27851724c6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:26.460305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-afed5931-dd38-4440-86ba-ec27851724c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:26.523849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 589/1168] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:26.545801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de21e-41fa-477a-946e-9fd63d66e757 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:26.607165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:26.644280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de21e-41fa-477a-946e-9fd63d66e757 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:26.677008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:26.677590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:26.677868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:26.678137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:26.678809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:26.680150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f5a87-5fca-411f-a055-b1fe71d52b38 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:26.695915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 580/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Wed Aug 10 11:02:26 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:28.691444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3da8-cfce-44fd-a660-9418a076837b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:28.691967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3da8-cfce-44fd-a660-9418a076837b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:28.691967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abd3da8-cfce-44fd-a660-9418a076837b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:28.693354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 590/1170] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:28.698316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:28.698745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:28.698892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:28.716637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:28.717793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:28.779746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 581/1171] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:28.803077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd3da8-cfce-44fd-a660-9418a076837b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:28.814947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:28.901548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abd3da8-cfce-44fd-a660-9418a076837b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:28.932901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:28.933508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:28.933785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:28.934054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] PATH_INFO: `/v3/domains/68b0e354d554487681bdd67f3c8a4cdb` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:28.943865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=68b0e354d554487681bdd67f3c8a4cdb)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:28.945289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638bb171-fd31-49c2-8704-4eb982733f14 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:28.946867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 591/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 10 11:02:28 2022] GET /identity/v3/domains/68b0e354d554487681bdd67f3c8a4cdb => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:28.953097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:29.004234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:29.047092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5476d77-d0dc-4be4-9a25-8cc9dc131786 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:29.078964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:29.079551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:29.079827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:29.080095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:29.080728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:29.082133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b957cf-e7c8-4ca1-b161-65ef7502c2e7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:29.106699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 582/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:02:28 2022] POST /identity/v3/projects => generated 361 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:31.096490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb59-45bc-478a-a227-878c3de636eb None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:31.097090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb59-45bc-478a-a227-878c3de636eb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:31.097090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06dbb59-45bc-478a-a227-878c3de636eb None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:31.098417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 592/1174] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:31.103463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:31.104355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:31.104544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:31.122536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:31.123678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:31.185842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 583/1175] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:31.209114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dbb59-45bc-478a-a227-878c3de636eb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:31.279107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:31.311053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06dbb59-45bc-478a-a227-878c3de636eb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:31.342189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:31.342820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:31.343098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:31.343368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] PATH_INFO: `/v3/domains/68b0e354d554487681bdd67f3c8a4cdb` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:31.344466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=68b0e354d554487681bdd67f3c8a4cdb)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:31.345870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4abcc0-6275-45db-a428-33b1b178e9cc None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:31.347492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 593/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Wed Aug 10 11:02:31 2022] GET /identity/v3/domains/68b0e354d554487681bdd67f3c8a4cdb => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:31.353068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:31.386352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:31.448768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02297cb-7da6-47c5-8877-6f5ada6e31c6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:31.481688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:31.482273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:31.482552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:31.482855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:31.483486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:31.484904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56631c5-3dea-4e1b-8413-c738a11b03a7 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:31.506106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 584/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:02:31 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:33.496962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485600b-0eba-457c-864c-e9f3c61e512e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:33.497485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485600b-0eba-457c-864c-e9f3c61e512e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:33.497485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485600b-0eba-457c-864c-e9f3c61e512e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:33.498859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 594/1178] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:33.503849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:33.504286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:33.504390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:33.522324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:33.523546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:33.587047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 585/1179] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:33.610246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485600b-0eba-457c-864c-e9f3c61e512e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:33.707018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485600b-0eba-457c-864c-e9f3c61e512e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:33.739013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:33.739612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:33.739888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:33.740156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:33.747172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:33.751440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:33.756562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:33.757765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:33.758861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:02:33.759775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 595/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:33.764077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:33.787488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:33.859937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19990056-ed56-4a44-a291-0960eb459b6d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:33.895859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:33.896460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:33.896738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:33.897007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:33.897662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:33.897962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:33.899348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:33.907015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 586/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:33.912783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:34.007587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32e67c-29b3-463d-8060-28b9c8ef69dc None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:34.039648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:34.040265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.040547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.040824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] PATH_INFO: `/v3/users/7dfae2eddbfc421f8e69ea97d4384de7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.053630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] RBAC: Authorizing `identity:get_user(user_id=7dfae2eddbfc421f8e69ea97d4384de7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:34.055085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:34.056649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 596/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:33 2022] GET /identity/v3/users/7dfae2eddbfc421f8e69ea97d4384de7 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:34.062817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:34.142338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:34.157427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53380c18-cddb-4993-8a7c-42a3ac7116cb None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:34.189617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:34.190202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.190480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.190783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.253338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:34.254796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:34.331800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:34.347473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 587/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:02:34.353326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:34.407030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:34.449595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0537929-66ef-456f-81a9-c211f523ac57 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:34.482719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:34.483329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.483607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.483876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] PATH_INFO: `/v3/projects/2b0c61d149c84fe3915d077868772341` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.484977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] RBAC: Authorizing `identity:get_project(project_id=2b0c61d149c84fe3915d077868772341)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:34.486380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa6a1f-f992-4c4e-8931-b054ebe0d890 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:34.487999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 597/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:34 2022] GET /identity/v3/projects/2b0c61d149c84fe3915d077868772341 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:34.493924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:34.586597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7140ddf-c703-4b05-adc9-567e54134f5c None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:34.618988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:34.619567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.619871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.620137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.620847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2b0c61d149c84fe3915d077868772341, user.id=7dfae2eddbfc421f8e69ea97d4384de7) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:34.621150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:34.622549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b4f2d-5e72-48d6-89e1-45aefdd0e13a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:34.627610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 588/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:34 2022] GET /identity/v3/role_assignments?user.id=7dfae2eddbfc421f8e69ea97d4384de7&scope.project.id=2b0c61d149c84fe3915d077868772341&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:36.586901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44fa3-b354-4c17-8d1e-4ac03081fc33 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:36.587453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44fa3-b354-4c17-8d1e-4ac03081fc33 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:36.587453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c44fa3-b354-4c17-8d1e-4ac03081fc33 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:36.589362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 598/1186] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:36.594596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:36.594999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:36.595143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:36.613222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:36.614412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:36.656273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:36.678691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 589/1187] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:36.701076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c44fa3-b354-4c17-8d1e-4ac03081fc33 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:36.796792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c44fa3-b354-4c17-8d1e-4ac03081fc33 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:36.819337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:36.830434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:36.831069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:36.831347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:36.831616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:36.841331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:36.845858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:36.846969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:02:36.848204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 599/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:36.852814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:36.947167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa6cf1-112b-437e-a4cf-117294049afb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:36.979228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:36.979822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:36.980101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:36.980374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:36.981026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:36.981326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:36.982712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:36.990292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 590/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:36.996432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:37.092041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4fc7-24fd-4a1c-b92f-4c56c4528504 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:37.124747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:37.125348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.125629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.125902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] PATH_INFO: `/v3/users/7dfae2eddbfc421f8e69ea97d4384de7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.127026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] RBAC: Authorizing `identity:get_user(user_id=7dfae2eddbfc421f8e69ea97d4384de7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:37.128441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:37.129969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 600/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:36 2022] GET /identity/v3/users/7dfae2eddbfc421f8e69ea97d4384de7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:37.135475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:37.175344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:37.230045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463f498-3424-4a16-9018-e34decd86dfe None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:37.261625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:37.262241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.262527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.262835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] PATH_INFO: `/v3/projects/2b0c61d149c84fe3915d077868772341` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.263943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] RBAC: Authorizing `identity:get_project(project_id=2b0c61d149c84fe3915d077868772341)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:37.265361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72969b-3a15-421e-b81b-d3d934149047 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:37.267032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 591/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:37 2022] GET /identity/v3/projects/2b0c61d149c84fe3915d077868772341 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:37.272683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:37.324281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:37.368256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0992a6e-e5f5-4573-9e8a-d71e2d2df4f2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:37.400310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:37.400940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.401247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.401519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] PATH_INFO: `/v3/projects/2b0c61d149c84fe3915d077868772341/users/7dfae2eddbfc421f8e69ea97d4384de7/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.403407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b0c61d149c84fe3915d077868772341, user_id=7dfae2eddbfc421f8e69ea97d4384de7, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:37.404803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a87a3-b9b0-430b-9d75-d14a53943243 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:37.416656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 601/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:02:37 2022] PUT /identity/v3/projects/2b0c61d149c84fe3915d077868772341/users/7dfae2eddbfc421f8e69ea97d4384de7/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:02:39.361838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01c6e9-7c15-4d76-b30d-1a64e8ed300b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.362388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01c6e9-7c15-4d76-b30d-1a64e8ed300b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.362388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01c6e9-7c15-4d76-b30d-1a64e8ed300b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.363751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 592/1193] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:39.368858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.369273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.369407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.387566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:39.388732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:39.451524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 602/1194] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:02:39.474769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01c6e9-7c15-4d76-b30d-1a64e8ed300b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:39.570030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01c6e9-7c15-4d76-b30d-1a64e8ed300b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:39.579152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:39.602891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:39.603490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.603772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.604041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.613680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:39.618158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:39.619386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:02:39.620475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 593/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:39.625251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:39.667862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:39.725668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729cc30-8f3a-4dda-963a-c77cd9587ce5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:39.758019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:39.758635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.758918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.759188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.759849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:39.760152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:39.762093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:39.769802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 603/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:39.775828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:39.872992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe71fe43-a386-4125-bb6d-7df90713a752 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:39.905815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:39.906443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.906783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.907172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] PATH_INFO: `/v3/users/7dfae2eddbfc421f8e69ea97d4384de7` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.908350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] RBAC: Authorizing `identity:get_user(user_id=7dfae2eddbfc421f8e69ea97d4384de7)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:39.909773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:39.911520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 594/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:39 2022] GET /identity/v3/users/7dfae2eddbfc421f8e69ea97d4384de7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:39.917178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.014034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aafb3c-4a1c-406b-8bdd-10f2c30b4d46 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.016308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.046141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.046768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.047055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.047329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.111374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.113015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.204720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 604/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:02:40.210525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.245703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.308287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb447479-5fbe-4f3a-aebf-d3ab1b2da303 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.340066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.340704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.340986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.341262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] PATH_INFO: `/v3/projects/2b0c61d149c84fe3915d077868772341` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.342369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] RBAC: Authorizing `identity:get_project(project_id=2b0c61d149c84fe3915d077868772341)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.343822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360104c8-6977-4edd-a1b7-9f77a6c9f6fe None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.345430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 595/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/projects/2b0c61d149c84fe3915d077868772341 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:40.351513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.354996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.445391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3e57ff-b6ed-4af3-b9ae-66f34be75b21 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.477356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.477880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.478157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.478422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.479169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2b0c61d149c84fe3915d077868772341, user.id=7dfae2eddbfc421f8e69ea97d4384de7) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:40.480004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.481394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525ae63d-0dfd-4ae7-a929-ba5a3a704402 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.486847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 605/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/role_assignments?user.id=7dfae2eddbfc421f8e69ea97d4384de7&scope.project.id=2b0c61d149c84fe3915d077868772341&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:50.335931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=48 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:50.484356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=66 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.345207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02317-b9b8-4816-b9f0-908d87fcedad None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.345814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02317-b9b8-4816-b9f0-908d87fcedad None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.345814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c02317-b9b8-4816-b9f0-908d87fcedad None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.348959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 596/1201] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:35.356400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.356865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.356959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.380551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:35.382050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:35.479952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 606/1202] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:03:35.509568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c02317-b9b8-4816-b9f0-908d87fcedad None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.655546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c02317-b9b8-4816-b9f0-908d87fcedad None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.704418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.705061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.705346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.705684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.720201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:35.722182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:35.741660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:35.743096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:35.744171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:35.744171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:35.744171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:35.746721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 597/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:35.749631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.895066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdac3d0-7f21-456a-9995-2f768d7e41a3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.929919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.938064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.938802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.939114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.939454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.940091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:35.940459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:35.941829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa06d80-235c-4fe6-bf74-7d4c284f43c0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:35.958603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 607/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:35.960286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.027157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.091673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83f08-036b-4193-a92a-3d3386a3f697 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.137718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.138342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.138654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.138932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.139586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.141092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2bc88-83d6-4e5c-8731-4dcbfa00514b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.179726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 598/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:35 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:38.380555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b762-d687-47b4-ad5d-b4e9761b40d9 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.381200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b762-d687-47b4-ad5d-b4e9761b40d9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.381200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241b762-d687-47b4-ad5d-b4e9761b40d9 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.382478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 608/1206] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:38.388128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.388600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.388688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.409275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:38.410795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:38.494495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 599/1207] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:03:38.520501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241b762-d687-47b4-ad5d-b4e9761b40d9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.640577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241b762-d687-47b4-ad5d-b4e9761b40d9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.685280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.685948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.686252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.686598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.704701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:38.709802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:38.711102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:38.712676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:38.714735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 609/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:38.718816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.763584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.838037 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecb92a7-502a-4dd7-9c18-5ee49883abce None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.879375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.880286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.880720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.881178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.882098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:38.882455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:38.884519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:38.893089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 600/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:38.899063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.960752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.019716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b91f3-7ad9-4fec-a924-20fb3f8f3d37 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.062377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.063108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.063462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.063776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.076686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.078296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:39.095439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:39.096622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:39.097559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:39.097559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:39.097559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:39.097559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 610/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:39.102291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.214705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.217918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e07fffa-595c-40f8-856f-de8fce9fc2e0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.253040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.253690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.254008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.254285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.255049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:39.255317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.256745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.269480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 601/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:39.275240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.387164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bdaf0a-4bc1-46a2-a9dd-e09469eb66a5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.388776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.432280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.433240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.433724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.434197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.445302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.447166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:39.461033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:39.462290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:39.463657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 611/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:39.466668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.597409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299727a8-e711-4baa-9c2b-2b062683e641 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.640880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.641491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.641772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.642044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.642814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:39.643265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.644713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.662992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 602/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:39.672253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.789581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f50a4-16a2-482e-9a65-413d4951fd70 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.831119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.831744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.832028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.832301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.872875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.917355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.919414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.050798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 612/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:03:40.051941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.072357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.168715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f275cd-bd1e-47f4-9a53-50b0830bb35f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.211378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.212687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.212976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.213276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.227686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.229434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:40.249536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:40.252047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:40.252949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:40.252949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:40.252949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:40.252949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 603/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:40.256344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.328449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.368897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94361409-0b0c-4b9c-9a38-6f1108e51843 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.410283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.410970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.411277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.411564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.412401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:40.412715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.414280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.428918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 613/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:40.435474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.510374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.549289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e904-5969-4a31-a94c-d2a93ddb7a41 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.595301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.595870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.596154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.596434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.604277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.606106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:40.620199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:40.621417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:40.622500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 604/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:40.629693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.750403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dec6e4-7d87-4338-b12a-413cd02878ea None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.776498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.790778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.791812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.792406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.792928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.794072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:40.794697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.796587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d951dd-ac5c-47a0-827e-69f079c6d4bd None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.812804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 614/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:40.819933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.937264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb82ec-28cb-444f-a354-421fbe53450e None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.976649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.984597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.985216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.985501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.985773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.986508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=05f5ea0d51694d8796b40d547d9a0474) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:40.986976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.988465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7f422-4650-453f-ad7f-34ae17a65629 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.002290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 605/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/role_assignments?user.id=05f5ea0d51694d8796b40d547d9a0474&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:43.040433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532772-1676-47dc-b9ee-65022de2e428 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.040971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532772-1676-47dc-b9ee-65022de2e428 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.041074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee532772-1676-47dc-b9ee-65022de2e428 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.042486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 615/1220] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:43.048230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.048659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.048752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.067625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:43.069077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:43.137328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 606/1221] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:03:43.159553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee532772-1676-47dc-b9ee-65022de2e428 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:43.269638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee532772-1676-47dc-b9ee-65022de2e428 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:43.304718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:43.305374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.305650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.305919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.315855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:43.320463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:43.321946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:43.322870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 616/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:43.328243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:43.427454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8918a1-ace8-460e-8cbf-da76ef7ccffd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:43.455013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:43.471350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:43.472338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.472809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.473276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.474427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] RBAC: Adding query filter params (name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:43.475285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:43.478276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:43.490270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 607/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:43.497137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:43.544712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:43.611774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa8295-b8f0-4519-8e77-874081cd38d4 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:43.646657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:43.647305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.647583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.647872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.659177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:43.661138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:43.676729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:43.677962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:43.678916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:43.678916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:43.678916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:43.678916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 617/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:43.683686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:43.784287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b94cd6-19b4-4763-a343-7118cd776136 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:43.818767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:43.819364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.819641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.819911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.820544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:43.821355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:43.822736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:43.835361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 608/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:43.842698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:43.943042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:43.950520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9275fcd-95fb-4643-b69c-201c59656965 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:43.984783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:43.985442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:43.985759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:43.986097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:43.994128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:43.995818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:44.008065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:44.009663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:44.010861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 618/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:44.016184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.064504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:44.120365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36964dc-a4fd-43ca-a3a8-06ea2ede6f71 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:44.157344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:44.157962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.158240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.158516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.159278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:44.159589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:44.161031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.178463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 609/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:03:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:44.185303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.299181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90619fbd-2556-436b-93e0-c6e15ab0b8e0 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:44.335264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:44.335900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.336187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.336487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.347803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:44.349312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:44.365959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:44.367311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:44.368747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:44.368747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:44.368747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:44.374620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 619/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:44.374863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.486117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7362d-dc24-42ae-8a98-71548cfbb6d4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:44.489052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:44.526903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:44.527610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.527867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.528131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.528802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:44.529170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:44.530668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.543951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 610/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:44.550068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.560689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:44.661375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659692bd-7ac3-4693-95bb-9d60465e1362 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:44.696562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:44.697208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.697477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.697744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.704525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:44.706021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:44.716826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:44.717999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:44.719128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 620/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:44.723934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.831748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6c141-99d6-4b5b-b7fb-5319a36d9718 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:44.865747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:44.866150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.866434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.866748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.867486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:44.868300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:44.869708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d29f215-0625-4b2d-904f-7c93182e65d6 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:44.883533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 611/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:44.889958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:44.964969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.004678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dad53c9-5ae1-41f0-96c2-7e1be3ecd992 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.042145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.042841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.043224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.043501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/05f5ea0d51694d8796b40d547d9a0474/roles/066c1748e0304dbba8e9c5ebf98c844e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.061005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=05f5ea0d51694d8796b40d547d9a0474, role_id=066c1748e0304dbba8e9c5ebf98c844e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.062432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4159281b-d742-47ac-88aa-73073d8b5094 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.076277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 621/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:44 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/05f5ea0d51694d8796b40d547d9a0474/roles/066c1748e0304dbba8e9c5ebf98c844e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:47.188280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43ad5-d537-4ae4-bdbd-3c118d1c26d8 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.188929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43ad5-d537-4ae4-bdbd-3c118d1c26d8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.188929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d43ad5-d537-4ae4-bdbd-3c118d1c26d8 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.190252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 612/1233] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:47.195622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.196032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.196173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.214628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:47.215847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:47.287648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 622/1234] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:03:47.317361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43ad5-d537-4ae4-bdbd-3c118d1c26d8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.345159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:47.429789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d43ad5-d537-4ae4-bdbd-3c118d1c26d8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.465839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.466467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.466799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.467045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.478920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:47.483701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:47.484870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:47.485742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 613/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:47.491956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.539877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:47.594762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36663990-336e-433b-974c-45b0d1ba4a51 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.631786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.632393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.633211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.633465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.634160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] RBAC: Adding query filter params (name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:47.634467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:47.635861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:47.643813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 623/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:47.650367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.737236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:47.754945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccceae9-7a6b-42ba-8b3e-754cfcf1a264 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.790777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.791470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.791768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.792165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.803254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:47.804743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:47.819936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:47.821662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:47.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:47.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:47.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:47.822611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 614/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:47.827732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.934867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03a4a-bb13-4adc-b365-e64c82097324 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.943976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:47.971908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.972553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.972870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.973150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.973809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:47.974105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:47.975505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:47.994882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 624/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:47.996127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.110600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498099ee-a95b-4a97-a31c-18954f0ab11b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.146004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.146651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.146934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.147203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.149812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.155053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:48.156502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:48.167778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:48.168814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:48.169956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 615/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:48.175141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.285539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5e683-465e-46c8-ba99-d3a018c4d1a5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.325428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.326173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.326454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.326749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.327488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:48.327832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:48.329327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:48.347580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 625/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:48.354027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.470761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688fd88-6ab0-4b17-a87b-1b749cdb5ffa None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.507087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.507722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.507995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.508275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.569758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.577287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:48.578839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:48.695302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 616/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:03:48.701393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.732810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.817985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45289735-c43a-41e0-ac15-aea3f4b3fe1d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.855548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.856106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.856387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.856654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.867090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:48.868581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:48.883436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:48.889387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:48.890614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:48.890614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:48.890614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:48.890614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 626/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:48.891069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.988477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.994696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5ca89-2174-469c-9053-8f1ad1b2f4f5 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.034008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.034663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.034946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.035218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.035870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.036173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.037529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.050209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 617/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:49.055925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.143491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.168388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be056c-699b-44a7-97fc-d57a0c54db18 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.203391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.204024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.204277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.204549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.212028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.214080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:49.225324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.226455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:49.227793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 627/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:49.232673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.341201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5af463-d46b-41da-814b-92010183c892 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.376283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.376942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.377160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.377436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.378132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.378439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.380034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc9c3f-9965-416a-ad4d-178d63f141f1 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.395666 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 618/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:49.401963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.520044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864e1e5-b609-46af-b43e-be4ef1d13c99 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.560506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.561167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.561492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.561766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.562615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] RBAC: Adding query filter params (role.id=066c1748e0304dbba8e9c5ebf98c844e, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=05f5ea0d51694d8796b40d547d9a0474) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.562954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.564356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd4a1d-eb67-4108-a669-0373f2da41e9 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.570517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 628/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/role_assignments?user.id=05f5ea0d51694d8796b40d547d9a0474&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=066c1748e0304dbba8e9c5ebf98c844e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:51.843409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc973f-e18a-4507-9493-ea3883d0cb95 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.843969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc973f-e18a-4507-9493-ea3883d0cb95 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.843969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc973f-e18a-4507-9493-ea3883d0cb95 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.845484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 619/1247] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:51.853771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.854309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.854309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.861243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.874309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:51.875530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:51.952479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 629/1248] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:03:51.980012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc973f-e18a-4507-9493-ea3883d0cb95 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.000025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.094934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc973f-e18a-4507-9493-ea3883d0cb95 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.129447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.130080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.130349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.130648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.141373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:52.146196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.147552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:52.149261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 620/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.154281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.267460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7022d6-95d1-4e5b-b3f8-e8393ba5126f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.302880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.308575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.309137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.309416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.309688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.310352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.310739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.312059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.320875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 630/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:52.327883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.417594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.442201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604ef2a-e3f5-4383-875d-e2023a96a7db None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.481424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.482177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.482359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.482692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.494766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.504507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:52.515823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.516858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:52.517983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:52.517983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:52.517983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:52.524153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 621/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.524319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.655017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67867df4-f75e-44d9-b0ec-cee6eeb10c04 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.696915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.697532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.697814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.698088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.698765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.699081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.700424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.714243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 631/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.719710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.828439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4ab6c-58b1-4b82-b892-7e3562c00916 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.869057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.878816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.879452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.879739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.880226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.891993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.894267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:52.906693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.907998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:52.909302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 622/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.915152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.974593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.025147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d6d6c-632d-4128-9737-6526dba93b3d None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.061212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.061812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.062088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.062357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.063113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.063416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.064809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.081247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 632/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.087171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.209253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd1c64-ad6e-4088-be4d-e7280c8cfc88 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.248380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.248975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.249258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.249533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.317227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.333371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.334854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.440477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 623/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:03:53.446366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.547535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524fa91-622e-4320-9ff8-3813b62fea8b None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.559993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.584533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.585782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.586104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.586656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.598166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.599701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:53.615523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:53.616984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:53.618019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:53.618019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:53.618019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:53.618019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 633/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:53.622453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.730287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.744270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ec6703-f28e-4e9e-adef-777165b69e6b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.780280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.780927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.781212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.781503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.782204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.782555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.783945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.798552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 624/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:53.804178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.911677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0081c456-7638-4f6e-bad5-5bb1b518464f None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.947183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.947947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.948203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.948550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.955947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.957512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:53.968792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:53.969862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:03:53.971176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 634/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:53.976962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.083827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbbc04-3781-47b0-88a2-609edf8a9b06 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.122507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.123267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.123547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.123818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.124498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.124808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.126428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ff502-6571-4030-91cf-16431ddf81f8 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.140827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 625/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:54.145437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.178432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.260854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aabd48-3345-4adc-9386-5bde5f3084a5 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.309201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.309934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.310340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.310647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.311439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=05f5ea0d51694d8796b40d547d9a0474) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.311746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.313189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e246f7a0-7495-48ec-af46-21bb1b749db8 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.319990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 635/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/role_assignments?user.id=05f5ea0d51694d8796b40d547d9a0474&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:56.335733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a35e-b1b7-4d8d-b7f4-54c3a95b5ad8 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.336260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a35e-b1b7-4d8d-b7f4-54c3a95b5ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.336260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781a35e-b1b7-4d8d-b7f4-54c3a95b5ad8 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.337624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 626/1261] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:56.342749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.343144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.343302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.361275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:56.362445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:56.425621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 636/1262] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:03:56.447102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781a35e-b1b7-4d8d-b7f4-54c3a95b5ad8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.457669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.547434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781a35e-b1b7-4d8d-b7f4-54c3a95b5ad8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.580567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.581207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.581489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.581747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.593690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:56.598864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.599986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:56.601343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 627/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.606384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.627955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.709579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e80b3-cf2c-47f6-97b4-0e5c080a5304 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.741678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.742271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.742547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.742862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.743482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.743782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.745161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.752780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 637/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:56.759074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.822840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.854932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3343ef0-c85a-4554-91ae-57a6b0f9920a None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.888906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.889536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.889818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.890091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.900817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.902248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:56.916102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.917181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:56.918207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:56.918207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.918207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:56.918207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 628/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.923231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.001301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.020635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa25ecb-a69f-4b31-8aae-9cd936f43b51 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.051988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.052581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.052856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.053122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.054250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.054546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.055911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.067677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 638/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.073887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.168349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c542809-2242-41bd-98a4-ce5bf63c2420 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.187378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.201362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.201965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.202246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.202514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] PATH_INFO: `/v3/users/placement` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.209512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.210988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:57.221441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.222510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:57.223697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 629/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.228086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.326092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d3d5-4cd7-41ce-b2f9-8fad1f4477f2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.358324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.358960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.359212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.359479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.360200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.360501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.361895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.370860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.379069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 639/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.384691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.479579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe213068-b0bd-4e19-97cd-c53d5e538faa None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.512054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.512665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.512946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.513216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.523198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.524631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:57.538402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.539558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:57.540635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:57.540635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:57.540635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:57.540635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 630/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.545162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.649437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be46d45-9cfa-46aa-8b27-e044418371d1 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.682616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.683216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.683497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.683770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.684405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.684707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.686050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.696753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 640/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.702410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.733705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.799280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf01dc-d904-4b8c-b027-97cfb68d3d55 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.830221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.830852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.831130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.831427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.838139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.839600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:57.850059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.851629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:03:57.852847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 631/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.857437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.914953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.956689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c590e3-9abd-4828-ac16-a9375edfb542 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.989218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.989747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.990025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.990295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.991039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.991346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.992740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63632eff-c08d-4b48-9c26-dd9e87d18343 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.004087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 641/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.010286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.097932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.106722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf71f0-c3bd-444f-935d-94f75264b5d4 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.138703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.139372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.139656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.139932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] PATH_INFO: `/v3/projects/2311045f60974b74910d615e2363e16d/users/05f5ea0d51694d8796b40d547d9a0474/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.141806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] RBAC: Authorizing `identity:create_grant(project_id=2311045f60974b74910d615e2363e16d, user_id=05f5ea0d51694d8796b40d547d9a0474, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.143267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb8aa7-f6a6-4c26-953c-48f93db7e771 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.155498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 632/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:58 2022] PUT /identity/v3/projects/2311045f60974b74910d615e2363e16d/users/05f5ea0d51694d8796b40d547d9a0474/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:00.155438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7349-eae3-4eb5-9066-0db9c00c2db5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.155936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7349-eae3-4eb5-9066-0db9c00c2db5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.155936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7349-eae3-4eb5-9066-0db9c00c2db5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.157237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 642/1274] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:00.162260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.163246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.163407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.181012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:00.182186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:00.245295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 633/1275] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:00.267765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc7349-eae3-4eb5-9066-0db9c00c2db5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.366065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc7349-eae3-4eb5-9066-0db9c00c2db5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.382787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.399685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.400298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.400577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.400850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.410543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:00.415153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.416238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:00.417239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 643/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.422399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.500548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.519250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921407c0-c14e-428b-8d87-0468fedeef6c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.551677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.552271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.552553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.552829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.553521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.553825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.555211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.562847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 634/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.568548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.674962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f28ec-287f-4928-a43e-a8c0bdb24945 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.707891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.708515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.708796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.709071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.718663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.720110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:00.733927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.735066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:00.736068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:00.736068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:00.736068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:00.736068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 644/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.741388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.837574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53552413-bedd-427a-9a5e-fdfabde5a2af None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.869540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.870137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.870418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.870722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.871357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.871662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.873011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.884297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 635/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.889920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.920059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.985858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba70082-0fb5-4373-b6d0-73cfa72a7bd2 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.018135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.018769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.019050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.019322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.026309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.027760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:01.038817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.044665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:01.045747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 645/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:01.046694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.049860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.141658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709db0f-ce8a-4002-b278-037aa22e550b None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.174468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.175092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.175375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.175652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.176376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.176683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.178676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.194650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 636/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.200279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.285652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.295499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd661f9-79d3-4349-b81c-6da567ec2572 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.327819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.328398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.328676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.328945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.392315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.393716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.477696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.487509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 646/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:01.493239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.552234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.594164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62856a0b-810b-45f3-b9e5-4aeaaf95128f None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.631187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.631806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.632087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.632357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.641812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.643261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:01.657701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.658743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:01.659665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:01.659665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.659665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:04:01.659665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 637/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:01.665029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.758182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36958408-1c0d-44e3-80e0-d8a52e9414de None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.790988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.791563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.791842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.792109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.792744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.793044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.794375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.805283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 647/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.810537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.907147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ad842-8ee3-4e14-90c7-7907a1dde6b0 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.919762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.939043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.939653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.939930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.940200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.946971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.948387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:01.958929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.960569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: ERROR keystone.server.flask.application Aug 10 11:04:01.961738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 638/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:01.966341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.013525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.061324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545275d-cc8b-4c23-a63c-37b88c21895e None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.093428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.094015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.094295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.094599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.095284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:02.095588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.097009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef015612-d066-4d29-ae65-0c3f1016035b None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.108349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 648/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:02.114332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.209400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0b5d9-68d0-4947-b987-659ea5a5a32d None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.240354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.241413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.241690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.241958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.242692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] RBAC: Adding query filter params (role.id=2eda6d42cc1048c1a4fd660987f7c37f, scope.project.id=2311045f60974b74910d615e2363e16d, user.id=05f5ea0d51694d8796b40d547d9a0474) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:02.242999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.244403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9629cb-2dbd-4fc8-9ae6-13620e99ba80 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.249789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 639/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/role_assignments?user.id=05f5ea0d51694d8796b40d547d9a0474&scope.project.id=2311045f60974b74910d615e2363e16d&role.id=2eda6d42cc1048c1a4fd660987f7c37f => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:04.290122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531971a-b81c-4652-a60b-a9cbb12ca11a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.290669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531971a-b81c-4652-a60b-a9cbb12ca11a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.290783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f531971a-b81c-4652-a60b-a9cbb12ca11a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.292031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 649/1288] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:04.297509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.297933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.298053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.316382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:04.317552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:04.349227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.385731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 640/1289] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:04.406994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531971a-b81c-4652-a60b-a9cbb12ca11a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.506915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f531971a-b81c-4652-a60b-a9cbb12ca11a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.517067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.541173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.541722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.542003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.542277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.542883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.544241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:04.552966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:04.554174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:04.555299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 650/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:04.559833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.667768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d338ee5-b802-4a45-aca3-9c5a31c8e83f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.699421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.700009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.700285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.700552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.701170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.701466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.702838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83068f1c-1d9b-4d35-b84b-3a4dd6017c1a None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.707405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 641/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:04.713650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.808861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959ee6a-a153-4f23-a831-16256e062ad6 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.841168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.841744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.842021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.842291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] PATH_INFO: `/v3/services` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.842938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.843242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.844582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc6466-0044-4ca1-a253-8528cc26efdb None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.848782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 651/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.803103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dffdb-21de-41cb-8899-a192886bd5d9 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.803893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dffdb-21de-41cb-8899-a192886bd5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.803893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0dffdb-21de-41cb-8899-a192886bd5d9 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.804985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 642/1293] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:06.810179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af2c2d-0b0f-45d2-ac27-ce5176d67f08 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.810596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af2c2d-0b0f-45d2-ac27-ce5176d67f08 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.810751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af2c2d-0b0f-45d2-ac27-ce5176d67f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.828929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e5af2c2d-0b0f-45d2-ac27-ce5176d67f08 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:06.830090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e5af2c2d-0b0f-45d2-ac27-ce5176d67f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:06.858270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.895896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 652/1294] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:06.918292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dffdb-21de-41cb-8899-a192886bd5d9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.948953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.015340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0dffdb-21de-41cb-8899-a192886bd5d9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.047829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.048413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.048689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.048957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.049587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.050959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc42ca-c43d-4441-8d17-b28bf8dfb272 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.064273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 643/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:06 2022] POST /identity/v3/services => generated 246 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:09.069678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068f31-0c21-4051-be66-fe6b0586967f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.070227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068f31-0c21-4051-be66-fe6b0586967f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.070227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51068f31-0c21-4051-be66-fe6b0586967f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.071592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 653/1296] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:09.077313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.078866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.078866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.095664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:09.096849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:09.166237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 644/1297] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:09.188950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068f31-0c21-4051-be66-fe6b0586967f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.286237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51068f31-0c21-4051-be66-fe6b0586967f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.319381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.319989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.320265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.320535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.321105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.322460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:09.331118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:09.332306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:09.333305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 654/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:09.338941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.356518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.434521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77723ed8-2dac-4e10-920a-7ee867549273 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.467921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.468512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.468796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.469071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.469700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:09.470005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.471392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.476799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 645/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:09.482596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.513749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.579079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef4917-5539-4a5b-9ecd-fc89bd4c5117 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.618831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.619437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.619720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.619988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.620690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] RBAC: Adding query filter params (interface=public, service_id=485183b79b97481d8cd1231c93b86ccb, region_id=RegionOne) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:09.621002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.622421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6016cf-4ddb-459d-a512-dbcb3e79f69a None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.627462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 655/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/endpoints?service_id=485183b79b97481d8cd1231c93b86ccb&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:09.634237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.713225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.731129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b145d-5975-4e4c-b07e-9793e3380001 None admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.762100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.762724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.763003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.763271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.763858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.765204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16de9234-1772-4f06-bbcd-ad6ce76a48fe None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.772281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 646/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:04:11.745680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446823-f249-4fac-9510-3b4118a9b6ae None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.746202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446823-f249-4fac-9510-3b4118a9b6ae None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.746202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b446823-f249-4fac-9510-3b4118a9b6ae None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.747578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 656/1302] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:11.752571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.752994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.753116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.771718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:11.772874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:11.835707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 647/1303] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:11.857663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b446823-f249-4fac-9510-3b4118a9b6ae None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:11.951575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:11.958095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b446823-f249-4fac-9510-3b4118a9b6ae None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:11.990349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:11.990991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.991271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.991539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.992118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:11.993465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:12.001847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:12.003016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: ERROR keystone.server.flask.application Aug 10 11:04:12.004272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 657/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:12.009563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.083996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:12.105975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e775c65-19ad-4cd2-9334-2fd1ce75c0b5 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.138418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.139036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.139316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.139584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] PATH_INFO: `/v3/services` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.140202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:12.140500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.141846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef967ec-c28e-4f84-95d7-84b2db27089e None admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.146716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 648/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:12.152409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.247732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b48d3d-0f16-4a5c-85fb-93bcce971b66 None admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.283667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.284197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.284477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.284749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.285410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.287278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1868f7-bb00-4698-85cc-89759af94ac7 None admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.307437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 658/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:22.143991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=30 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.302705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:14.522448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6775ad-55db-4502-9644-10945e272a19 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.523069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6775ad-55db-4502-9644-10945e272a19 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.523069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6775ad-55db-4502-9644-10945e272a19 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:14.528074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 649/1307] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:14.529375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b3ed-81da-46a6-8dcc-7a16135e8ed0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.529807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b3ed-81da-46a6-8dcc-7a16135e8ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.530024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908b3ed-81da-46a6-8dcc-7a16135e8ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:14.630232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c908b3ed-81da-46a6-8dcc-7a16135e8ed0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:14.631471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c908b3ed-81da-46a6-8dcc-7a16135e8ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:14.782954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 659/1308] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:05:14.838965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.839516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.840042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:14.842653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 650/1309] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 10 11:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:14.852759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7072eb-c379-4a7b-9d7d-26a78c9e87a5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.852759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7072eb-c379-4a7b-9d7d-26a78c9e87a5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.852759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7072eb-c379-4a7b-9d7d-26a78c9e87a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:15.002289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bc7072eb-c379-4a7b-9d7d-26a78c9e87a5 None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:15.004761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc7072eb-c379-4a7b-9d7d-26a78c9e87a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:15.039553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:15.111457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 660/1310] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 10 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:15.120768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:15.122506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:15.238115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fd926-16d6-4796-bd55-63f97be8ac3e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:15.313540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:15.314064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:15.314886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:15.315161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:15.316765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:15.373063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:15.434552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:15.436063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb5a6f-8dc0-4da5-b283-9787646ee0b3 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:15.459265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 651/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:17.765667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.766221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.766221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.772022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 661/1312] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:17.773450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fa22e-63cb-4ddb-a873-e3c5a5325e8e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.773932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fa22e-63cb-4ddb-a873-e3c5a5325e8e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.774016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6fa22e-63cb-4ddb-a873-e3c5a5325e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.868033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-5b6fa22e-63cb-4ddb-a873-e3c5a5325e8e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:17.869309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5b6fa22e-63cb-4ddb-a873-e3c5a5325e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:18.002562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 652/1313] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:18.041511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:18.043346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:18.154150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d385243-12e6-4394-900e-2b57ef1d2da5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:18.232097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:18.234960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:18.235664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:18.236176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:18.236631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:18.238429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:18.347693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:18.349052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae94ded-853d-42ad-95fb-4c6206859ba2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:18.367569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 662/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:20.687936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a352d-7945-4884-bc15-e460034660a6 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.688467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a352d-7945-4884-bc15-e460034660a6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.688467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57a352d-7945-4884-bc15-e460034660a6 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.689862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 653/1315] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:20.695159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037684-1f86-49ea-af1a-3d76b9255d7e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.695563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037684-1f86-49ea-af1a-3d76b9255d7e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.695713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb037684-1f86-49ea-af1a-3d76b9255d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.760796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-eb037684-1f86-49ea-af1a-3d76b9255d7e None None] MFA Rules not processed for user `26e85b5ffec84d34acc86041db6d5307`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:20.762007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eb037684-1f86-49ea-af1a-3d76b9255d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:20.813165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:20.856762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 663/1316] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:20.912296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.912587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.912862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.914202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 654/1317] 10.222.0.49 () {54 vars in 965 bytes} [Wed Aug 10 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:20.920465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c196b4-0c38-4049-8438-5c202e91f789 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.920746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c196b4-0c38-4049-8438-5c202e91f789 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.921014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c196b4-0c38-4049-8438-5c202e91f789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.993473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-56c196b4-0c38-4049-8438-5c202e91f789 None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:20.994697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-56c196b4-0c38-4049-8438-5c202e91f789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:21.067019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 664/1318] 10.222.0.49 () {58 vars in 1062 bytes} [Wed Aug 10 11:05:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:21.076321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:21.077872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:21.087844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:21.180591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337ed937-a8d0-4bdd-96b5-0d1afbff8af6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:21.251447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:21.252481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:21.252762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:21.253031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:21.254667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:21.319013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:21.364548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:21.365987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ed940-1604-4a45-8f89-c4462562d569 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:21.384472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 655/1319] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:24.300628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.301169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.301169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.302520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 665/1320] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:24.307669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5e5e8-9741-4297-823f-d9c63b11008e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.308053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5e5e8-9741-4297-823f-d9c63b11008e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.308245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5e5e8-9741-4297-823f-d9c63b11008e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.372340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-28d5e5e8-9741-4297-823f-d9c63b11008e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:24.373537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-28d5e5e8-9741-4297-823f-d9c63b11008e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:24.467927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 656/1321] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:05:24.506824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:24.508332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:24.557978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.613134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657fef3-7f5d-4a25-9e0e-b88cd95eeaf9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:24.685076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:24.685673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.685955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.686230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.687854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:24.781901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.795425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:24.796852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbbc698-bdd1-415b-8ce6-517f9509fd39 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.815368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 666/1322] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:27.491801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.492406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.492406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.493686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 657/1323] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:27.498809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7581-9288-407c-8c5c-8a363aa63808 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.499201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7581-9288-407c-8c5c-8a363aa63808 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.499368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e7581-9288-407c-8c5c-8a363aa63808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.565031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-274e7581-9288-407c-8c5c-8a363aa63808 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:27.566225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-274e7581-9288-407c-8c5c-8a363aa63808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:27.670462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 667/1324] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:27.708083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:27.709649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:27.729622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:27.820254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d8e14-6fe3-4442-bae4-cdb49a1dcb57 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:27.892702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:27.893312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.893595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.893869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.895498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:27.964475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:28.001961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:28.003418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3685dc6-1020-468e-a028-c9f5dade629d service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:28.021064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 658/1325] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:30.074148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:30.074708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:30.074708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:30.076053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 668/1326] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:30.081373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d447a8-2fc3-43ba-81d5-0b4ba729f720 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:30.081791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d447a8-2fc3-43ba-81d5-0b4ba729f720 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:30.081916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d447a8-2fc3-43ba-81d5-0b4ba729f720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:30.146439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-40d447a8-2fc3-43ba-81d5-0b4ba729f720 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:30.147558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-40d447a8-2fc3-43ba-81d5-0b4ba729f720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:30.241921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 659/1327] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:30.278294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:30.279841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:30.340708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:30.384065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acebdd6-89c2-49db-95da-e8cb6addd5e1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:30.460264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:30.460858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:30.461135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:30.461403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:30.463021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:30.572843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:30.576367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:30.579154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec21cda-d70a-406a-89ba-7fdd76e2e01a service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:30.598583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 669/1328] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:05:32.678109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:32.678658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:32.678754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:32.680197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 660/1329] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:32.685629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a820e-5675-4211-b199-d9acbde83b55 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:32.686030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a820e-5675-4211-b199-d9acbde83b55 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:32.686196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a820e-5675-4211-b199-d9acbde83b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:32.751722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e82a820e-5675-4211-b199-d9acbde83b55 None None] MFA Rules not processed for user `26e85b5ffec84d34acc86041db6d5307`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:32.752928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e82a820e-5675-4211-b199-d9acbde83b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:32.851490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 670/1330] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:32.889080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:32.890710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:32.926395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:33.000844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ddbe8e-1f7a-4fd5-ae6c-dd704ed76f7b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:33.078386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:33.079017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:33.079811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:33.080080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:33.081654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:33.159026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:33.191454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:33.192894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1934e-907d-4222-be8f-76ce2d6afca1 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:33.212297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 661/1331] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:35.537485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:35.538042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:35.538042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:35.539442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 671/1332] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:35.544680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735d99-3dc2-4793-99aa-cf2847530ddb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:35.545214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735d99-3dc2-4793-99aa-cf2847530ddb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:35.545214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5735d99-3dc2-4793-99aa-cf2847530ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:35.609466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e5735d99-3dc2-4793-99aa-cf2847530ddb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:35.611265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e5735d99-3dc2-4793-99aa-cf2847530ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:35.706958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 662/1333] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:35.743198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:35.744931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:35.812507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:35.845829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09869f-bc44-46b7-a6e9-72185bb84512 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:35.915844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:35.916442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:35.916721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:35.916990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:35.918847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:36.024689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:36.026113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f659d24-c40a-4041-a911-748446e8010e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:36.035880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:36.044770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 672/1334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:38.068420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:38.068942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:38.068942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:38.070306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 663/1335] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:38.075507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b52-4644-4dd2-a049-d0c95422aafc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:38.075903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b52-4644-4dd2-a049-d0c95422aafc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:38.076060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b52-4644-4dd2-a049-d0c95422aafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:38.141936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b19c5b52-4644-4dd2-a049-d0c95422aafc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:38.143099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b19c5b52-4644-4dd2-a049-d0c95422aafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:38.236967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 673/1336] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:38.277308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:38.278848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:38.318483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:38.381706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f04e2c-fc31-45b4-8f7c-720cddda9af4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:38.451818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:38.452417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:38.452692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:38.452957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:38.455077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:38.535423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:38.560040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:38.561456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fd1b4-661b-420d-8a5a-8a302569d3f0 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:38.579649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 664/1337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:41.024213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:41.024732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:41.024827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:41.026108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 674/1338] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:05:41.031157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2a03-1c47-4f5a-a975-6d0009349331 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:41.031583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2a03-1c47-4f5a-a975-6d0009349331 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:41.031680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a2a03-1c47-4f5a-a975-6d0009349331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:41.096935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9e6a2a03-1c47-4f5a-a975-6d0009349331 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:41.098135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9e6a2a03-1c47-4f5a-a975-6d0009349331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:41.191293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 665/1339] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:05:41.228139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:41.229679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:41.310353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:41.338280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825bec9-8881-447f-a709-a8bf9f4648fd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:41.409277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:41.409871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:41.410147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:41.410416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:41.412021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:41.517916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:41.519368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf6c3f-e817-4459-8589-c47a8bee0d34 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:41.538029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 675/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:45.664292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:45.664819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:45.664819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:45.666123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 666/1341] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:45.671796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b284-58fc-4988-a722-84279dcb1615 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:45.672228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b284-58fc-4988-a722-84279dcb1615 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:45.672371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b284-58fc-4988-a722-84279dcb1615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:45.677765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:45.738359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1899b284-58fc-4988-a722-84279dcb1615 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:45.739589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1899b284-58fc-4988-a722-84279dcb1615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:45.838684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 676/1342] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:45.878544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:45.880113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:45.928908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:45.984094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c05be9-2029-4ded-a82a-3ff1a82656fd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:46.057365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:46.057970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:46.058248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:46.058517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:46.060148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:46.155766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:46.173155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:46.174654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88e4a6-6196-4166-9dce-c38d92d8b3ad service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:46.194629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 667/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:48.829610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:48.830168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:48.830168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:48.831523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 677/1344] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:05:48.836875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fdc9-0128-4156-a7a4-c530a1beeda1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:48.837340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fdc9-0128-4156-a7a4-c530a1beeda1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:48.837438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457fdc9-0128-4156-a7a4-c530a1beeda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:48.904460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d457fdc9-0128-4156-a7a4-c530a1beeda1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:48.905670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d457fdc9-0128-4156-a7a4-c530a1beeda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:49.001206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 668/1345] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:49.037669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:49.039307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:49.125290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:49.143867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46817a00-a4a4-4cdf-ab87-8b9eb274a15b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:49.215796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:49.216395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:49.216670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:49.216938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:49.218512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:49.326433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:49.327862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67681652-015f-4d73-89bd-885c96627433 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:49.346154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 678/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:53.283878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:53.284407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:53.284688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:53.286247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 669/1347] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:53.291865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdc6e-a968-42e4-9b3e-7e1bc1a91be6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:53.292293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdc6e-a968-42e4-9b3e-7e1bc1a91be6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:53.292856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfdc6e-a968-42e4-9b3e-7e1bc1a91be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:53.309108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:53.365754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a5bfdc6e-a968-42e4-9b3e-7e1bc1a91be6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:53.367071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a5bfdc6e-a968-42e4-9b3e-7e1bc1a91be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:53.470607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 679/1348] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:05:53.512004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:53.513817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:53.576330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:53.625581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce23f70-c21f-4da5-9347-d61716dc2710 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:53.701784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:53.702400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:53.702711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:53.702991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:53.704642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:53.813961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:53.817841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:53.819497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bff41d-e946-4703-9592-531ec90bffb4 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:53.838629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 670/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:05:58.748656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29263f-61d7-4afd-bb72-60576b0ec92a None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:58.749499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29263f-61d7-4afd-bb72-60576b0ec92a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:58.749935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29263f-61d7-4afd-bb72-60576b0ec92a None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:58.752348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 680/1350] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:58.758795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a09e-c5f2-4f19-83a6-a61f717e56c8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:58.759618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a09e-c5f2-4f19-83a6-a61f717e56c8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:58.760073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6a09e-c5f2-4f19-83a6-a61f717e56c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:58.845409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ddb6a09e-c5f2-4f19-83a6-a61f717e56c8 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:58.847335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ddb6a09e-c5f2-4f19-83a6-a61f717e56c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:59.005284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 671/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:05:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:02.998211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff0d3-9c04-4d93-b8df-f5c8907b67a7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:02.999090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff0d3-9c04-4d93-b8df-f5c8907b67a7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:02.999369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ff0d3-9c04-4d93-b8df-f5c8907b67a7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:03.001763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 681/1352] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:03.007444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e00ed-f44d-4e2b-9aa2-dde9ef8fbbe7 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:03.007838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e00ed-f44d-4e2b-9aa2-dde9ef8fbbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:03.008034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e00ed-f44d-4e2b-9aa2-dde9ef8fbbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:03.066147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:03.089525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-652e00ed-f44d-4e2b-9aa2-dde9ef8fbbe7 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:03.090818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-652e00ed-f44d-4e2b-9aa2-dde9ef8fbbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:03.201918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 672/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:06:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:03.468270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=59 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:04.597356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1988f-877d-43f0-ba9f-d056b4172967 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:04.597944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1988f-877d-43f0-ba9f-d056b4172967 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:04.597944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1988f-877d-43f0-ba9f-d056b4172967 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:04.599171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 682/1354] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:04.604089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc9cb-04f5-4cb0-87bc-b5d4fe64c69d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:04.604854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc9cb-04f5-4cb0-87bc-b5d4fe64c69d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:04.605138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079bc9cb-04f5-4cb0-87bc-b5d4fe64c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:04.683159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-079bc9cb-04f5-4cb0-87bc-b5d4fe64c69d None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:04.684360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-079bc9cb-04f5-4cb0-87bc-b5d4fe64c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:04.726871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:04.796091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 673/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:08.210918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1621b-7e07-4c6d-9a7b-585752e0eb63 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:08.211725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1621b-7e07-4c6d-9a7b-585752e0eb63 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:08.212157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1621b-7e07-4c6d-9a7b-585752e0eb63 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:08.214372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 683/1356] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:08.220079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08febf-936e-47d7-a8dd-9a2ef16e2113 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:08.221739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08febf-936e-47d7-a8dd-9a2ef16e2113 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:08.223162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08febf-936e-47d7-a8dd-9a2ef16e2113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:08.302259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-bd08febf-936e-47d7-a8dd-9a2ef16e2113 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:08.303799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd08febf-936e-47d7-a8dd-9a2ef16e2113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:08.406515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 674/1357] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:08.468667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:08.468951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:08.470108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:08.471725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 684/1358] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 11:06:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:08.479936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7bbe-66fa-44a7-8ef7-45e161576d8a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:08.480506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7bbe-66fa-44a7-8ef7-45e161576d8a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:08.480643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d7bbe-66fa-44a7-8ef7-45e161576d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:08.499756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:08.563939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9b4d7bbe-66fa-44a7-8ef7-45e161576d8a None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:08.565122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9b4d7bbe-66fa-44a7-8ef7-45e161576d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:08.703144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 675/1359] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:08.708231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:08.710229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:08.871306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a389c1b-a0f9-4e3c-bb74-a6f9e033e150 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:08.950408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:08.994429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:08.994978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:08.995254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:08.995520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:08.997082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:09.099566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:09.101279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df84894-89b6-4ed7-99cc-dd1f0f8865cd service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:09.120068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 685/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:11.330051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:11.330633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:11.330633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:11.332122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 676/1361] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:11.337101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296b02-c96d-45f5-b091-ea990b9bebe3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:11.337489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296b02-c96d-45f5-b091-ea990b9bebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:11.337658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15296b02-c96d-45f5-b091-ea990b9bebe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:11.406084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-15296b02-c96d-45f5-b091-ea990b9bebe3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:11.408017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-15296b02-c96d-45f5-b091-ea990b9bebe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:11.447062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:11.512009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 686/1362] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:11.559284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:11.560818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:11.641382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:11.737021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92a49c-da95-42c1-bd3a-1f8b3a091f5c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:11.865168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:11.865822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:11.865822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:11.866277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:11.868249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:11.894616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:11.981988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:11.983621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0a4ad-ecb5-481c-b1fe-adaa17c29d9b service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:12.002662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 677/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:14.374747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e8b2-f7bc-42d7-a4b7-ead57046d4a5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:14.375629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e8b2-f7bc-42d7-a4b7-ead57046d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:14.375904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a660e8b2-f7bc-42d7-a4b7-ead57046d4a5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:14.377246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 687/1364] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:14.382761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456783-bb33-42ed-b5e9-16a6d64e6164 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:14.383196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456783-bb33-42ed-b5e9-16a6d64e6164 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:14.383321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f456783-bb33-42ed-b5e9-16a6d64e6164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:14.461189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-8f456783-bb33-42ed-b5e9-16a6d64e6164 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:14.462392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8f456783-bb33-42ed-b5e9-16a6d64e6164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:14.576082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:14.579893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 678/1365] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:14.626358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:14.627142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:14.627633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:14.628958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 688/1366] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 11:06:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:14.634965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a56a1-81c3-4e81-947c-93fd98d2b4d8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:14.635427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a56a1-81c3-4e81-947c-93fd98d2b4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:14.635699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a56a1-81c3-4e81-947c-93fd98d2b4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:14.716580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f43a56a1-81c3-4e81-947c-93fd98d2b4d8 None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:14.717679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f43a56a1-81c3-4e81-947c-93fd98d2b4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:14.840650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 679/1367] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:14.848850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:14.850827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:15.012722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058ceb6-5a01-4da8-abc7-64759757521f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:15.039957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:15.136987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:15.137591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:15.137872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:15.138154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:15.139780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:15.233934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:15.235630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3c720-3440-43b2-814c-f270e2130bce service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:15.255085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 689/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:16.322960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f06c1-1896-4393-977a-5e45aa353ab2 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:16.323518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f06c1-1896-4393-977a-5e45aa353ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:16.323518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f06c1-1896-4393-977a-5e45aa353ab2 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:16.325091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 680/1369] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:16.330541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff6a4-b5bd-4d1a-b083-cb5a4b25c3ae None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:16.331009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff6a4-b5bd-4d1a-b083-cb5a4b25c3ae None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:16.331102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ff6a4-b5bd-4d1a-b083-cb5a4b25c3ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:16.367891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:16.404251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d02ff6a4-b5bd-4d1a-b083-cb5a4b25c3ae None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:16.405467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d02ff6a4-b5bd-4d1a-b083-cb5a4b25c3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:16.507563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 690/1370] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:16.876192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:16.876192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:16.876192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:16.877407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 681/1371] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 10 11:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:16.882286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901c2-c0e9-4c30-ac3d-087e53f93e6b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:16.882486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901c2-c0e9-4c30-ac3d-087e53f93e6b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:16.882787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707901c2-c0e9-4c30-ac3d-087e53f93e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:16.964090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-707901c2-c0e9-4c30-ac3d-087e53f93e6b None None] MFA Rules not processed for user `5882280742154859b52fa47c350a137f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:16.965402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-707901c2-c0e9-4c30-ac3d-087e53f93e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:17.021375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:17.060508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 691/1372] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 10 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:17.068740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:17.070296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:17.128471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:17.188614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9168df-ab23-4bc2-a878-cdd31027d75f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:17.264726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:17.265500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:17.265791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:17.266098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:17.267736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:17.372453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:17.388382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:17.389872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04a4d8-1b99-4fa7-a15b-6974c489700f service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:17.410118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 682/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:17.474895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:17.475366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:17.475477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:17.476841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 692/1374] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:17.481819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949210-d500-4b4f-8b15-6b115112f3ba None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:17.482059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949210-d500-4b4f-8b15-6b115112f3ba None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:17.482333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e949210-d500-4b4f-8b15-6b115112f3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:17.559073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-5e949210-d500-4b4f-8b15-6b115112f3ba None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:17.561209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5e949210-d500-4b4f-8b15-6b115112f3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:17.675212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 683/1375] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:17.719857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:17.720743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:17.881297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb973a7-263c-4e82-8318-cf1156b2e3fc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:17.931931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:17.994852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:17.995859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:17.996367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:17.996852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:17.998636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:18.106798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:18.108405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cfa664-b348-4ee5-9d6e-7990ce0d5596 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:18.127730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 693/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:20.210102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:20.210538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:20.210916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:20.212360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 684/1377] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:20.223658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aebf-3b72-4525-a53e-51b528408485 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:20.223658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aebf-3b72-4525-a53e-51b528408485 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:20.223658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6aebf-3b72-4525-a53e-51b528408485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:20.284943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-93f6aebf-3b72-4525-a53e-51b528408485 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:20.286147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-93f6aebf-3b72-4525-a53e-51b528408485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:20.293065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:20.406645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 694/1378] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:20.446078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:20.447678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:20.508491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:20.594754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6babceb2-0bb1-4b27-a54a-fe30260b4dd4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:20.713756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:20.714361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:20.714676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:20.714952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:20.716544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:20.737746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:20.825665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:20.827715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fbad3-afbe-42bd-a3c8-a4222b852931 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:20.853124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 685/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:22.962886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:22.963426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:22.963426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:22.965135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 695/1380] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:22.970698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec32fd-2c83-40a2-85f9-54b5158e5f39 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:22.971097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec32fd-2c83-40a2-85f9-54b5158e5f39 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:22.971277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec32fd-2c83-40a2-85f9-54b5158e5f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:23.041739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a2ec32fd-2c83-40a2-85f9-54b5158e5f39 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:23.043226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a2ec32fd-2c83-40a2-85f9-54b5158e5f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:23.138714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:23.150708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 686/1381] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:23.191430 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:23.192930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:23.341091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:23.353099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757806e9-e97f-4496-8a82-37df36df9d5d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:23.471295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:23.471931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:23.472222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:23.472490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:23.474058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:23.568397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:23.569986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b0f0b-fa0f-48ca-ab15-9cdb9c4db80d service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:23.580672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:23.604652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 696/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:25.811741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:25.812345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:25.812474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:25.814832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 687/1383] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:25.820125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999d338-08a0-41e6-8ab3-8f6b5c8b7fd6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:25.820612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999d338-08a0-41e6-8ab3-8f6b5c8b7fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:25.820772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999d338-08a0-41e6-8ab3-8f6b5c8b7fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:25.886896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d999d338-08a0-41e6-8ab3-8f6b5c8b7fd6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:25.888105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d999d338-08a0-41e6-8ab3-8f6b5c8b7fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:25.989653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 697/1384] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:26.032962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:26.034873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:26.198139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259f6e55-595e-4338-bbe8-5f73c7fc4b52 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:26.268310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:26.325372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:26.326114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:26.326531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:26.326901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:26.329001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:26.420671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:26.422985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513aa0c6-e722-46dc-961c-fa18206cc139 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:26.441816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 688/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:28.450306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:28.450888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:28.451383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:28.453803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 698/1386] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:28.459240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480584dd-17b8-4b64-9742-1ad192fbe8ad None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:28.459810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480584dd-17b8-4b64-9742-1ad192fbe8ad None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:28.459810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480584dd-17b8-4b64-9742-1ad192fbe8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:28.524155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-480584dd-17b8-4b64-9742-1ad192fbe8ad None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:28.525320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-480584dd-17b8-4b64-9742-1ad192fbe8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:28.558591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:28.630391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 689/1387] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:06:28.678601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:28.680139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:28.761577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:28.834105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ef617-aa5c-4dfa-a19f-2b6270a8c54b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:28.966557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:28.967396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:28.967746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:28.968109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:28.970774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:29.016695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:29.075467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:29.077361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b74c1a-4a78-448d-8983-c1be6bf1fd75 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:29.097547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 699/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:31.293010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:31.293549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:31.293549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:31.294951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 690/1389] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:31.300322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7ca9-ab08-40ff-aa0a-df9249564c14 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:31.300742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7ca9-ab08-40ff-aa0a-df9249564c14 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:31.300845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ab7ca9-ab08-40ff-aa0a-df9249564c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:31.366214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e0ab7ca9-ab08-40ff-aa0a-df9249564c14 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:31.367433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e0ab7ca9-ab08-40ff-aa0a-df9249564c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:31.461647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 700/1390] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:31.505580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:31.507110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:31.651815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a61b8e-8220-4ee2-a1fd-6b810863581f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:31.668889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:31.765533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:31.766719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:31.767002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:31.767272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:31.769105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:31.858319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:31.860021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993b6c6-1c37-4847-b2f2-c48163709eeb service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:31.885602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 691/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:32.817318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222849-2fab-4c11-bd77-562d81e32610 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:32.817969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222849-2fab-4c11-bd77-562d81e32610 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:32.817969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6222849-2fab-4c11-bd77-562d81e32610 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:32.819323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 701/1392] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:32.824826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd0d6-395b-4fcb-a4e9-b2243fef34b2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:32.825396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd0d6-395b-4fcb-a4e9-b2243fef34b2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:32.825396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecd0d6-395b-4fcb-a4e9-b2243fef34b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:32.852242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:32.898551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-efecd0d6-395b-4fcb-a4e9-b2243fef34b2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:32.899791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-efecd0d6-395b-4fcb-a4e9-b2243fef34b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:32.995635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 692/1393] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:33.053141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:33.059134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:33.059134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:33.059134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 702/1394] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 10 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:33.060472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32451f05-9366-449e-853e-3c45c5fe2566 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:33.060766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32451f05-9366-449e-853e-3c45c5fe2566 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:33.061042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32451f05-9366-449e-853e-3c45c5fe2566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:33.257980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-32451f05-9366-449e-853e-3c45c5fe2566 None None] MFA Rules not processed for user `eacd5dcb5de7474da98c2e7f30a4e6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:33.259215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-32451f05-9366-449e-853e-3c45c5fe2566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:33.300186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:33.336127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 693/1395] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 10 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:33.343611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:33.345664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:33.359225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:33.450790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc94dca6-a443-48af-b13a-1105167423a7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:33.524786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced53dc-4988-451e-9782-328f48744551 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:33.525308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced53dc-4988-451e-9782-328f48744551 service glance] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:33.525586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced53dc-4988-451e-9782-328f48744551 service glance] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:33.525857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced53dc-4988-451e-9782-328f48744551 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:33.527504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dced53dc-4988-451e-9782-328f48744551 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:33.597016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:33.639121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced53dc-4988-451e-9782-328f48744551 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:33.641147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced53dc-4988-451e-9782-328f48744551 service glance] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:33.659532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 703/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:06:33.901157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:33.901726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:33.901726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:33.903121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 694/1397] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:33.909455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce695-11ab-42c8-9466-13aaa47987a0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:33.910034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce695-11ab-42c8-9466-13aaa47987a0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:33.910034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cce695-11ab-42c8-9466-13aaa47987a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:33.984107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-20cce695-11ab-42c8-9466-13aaa47987a0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:33.985342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-20cce695-11ab-42c8-9466-13aaa47987a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.089418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 704/1398] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:34.133699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:34.135335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.296355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe87ab98-9171-4ac5-b5c3-fd5fe615555e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:34.346651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:34.397967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917a11-c0a3-4d62-a837-29976c081633 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:34.398348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917a11-c0a3-4d62-a837-29976c081633 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:34.399138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6917a11-c0a3-4d62-a837-29976c081633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:34.415693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:34.416295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:34.416574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:34.416851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:34.418978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.427402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:34.468764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b6917a11-c0a3-4d62-a837-29976c081633 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:34.470539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b6917a11-c0a3-4d62-a837-29976c081633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.508348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:34.509978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8f824-c806-4472-983a-615bd50c7bbe service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:34.528850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 695/1399] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:34.571692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 705/1400] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:06:34.618300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:34.618608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:34.618890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:34.620408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 696/1401] 10.222.0.49 () {54 vars in 943 bytes} [Wed Aug 10 11:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:34.627593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d136bb-e8ba-4bae-9847-4abec92e9a18 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:34.627885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d136bb-e8ba-4bae-9847-4abec92e9a18 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:34.628195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d136bb-e8ba-4bae-9847-4abec92e9a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:34.712487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-06d136bb-e8ba-4bae-9847-4abec92e9a18 None None] MFA Rules not processed for user `70fc784da233409a9fa518088a4ac68d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:34.713740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-06d136bb-e8ba-4bae-9847-4abec92e9a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.769780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:34.823176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 706/1402] 10.222.0.49 () {58 vars in 1040 bytes} [Wed Aug 10 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:34.833067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:34.834596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:34.891870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:34.970926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a68c72-82a9-45b1-a2ba-bbefd601ac70 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:35.074414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:35.075068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:35.075341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:35.075610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:35.077219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:35.113867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:35.161157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:35.162674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503d6ee-fa36-4da5-9c44-4aefe5931332 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:35.170813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 697/1403] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:06:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:06:36.590756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:36.591560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:36.591560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:36.593138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 707/1404] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:36.599454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13b0-ad65-49e8-8ec0-703d0e2b77a1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:36.599884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13b0-ad65-49e8-8ec0-703d0e2b77a1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:36.599989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c13b0-ad65-49e8-8ec0-703d0e2b77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:36.675226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a94c13b0-ad65-49e8-8ec0-703d0e2b77a1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:36.676435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a94c13b0-ad65-49e8-8ec0-703d0e2b77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:36.782627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 698/1405] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:36.825737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:36.827265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:37.024448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0674c3-fe8e-4c20-8fe6-f210a6b653aa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:37.056781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:37.153271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:37.153796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:37.154070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:37.154333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:37.155971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:37.264456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:37.266326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efff1f-5cc2-4c44-abe3-21bba8d47938 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:37.290544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 708/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:39.497190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:39.497742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:39.497742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:39.499425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 699/1407] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:39.504878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283d531-7f7f-4d37-824f-ac5c5965af31 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:39.505281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283d531-7f7f-4d37-824f-ac5c5965af31 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:39.505445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283d531-7f7f-4d37-824f-ac5c5965af31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:39.549893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:39.577725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7283d531-7f7f-4d37-824f-ac5c5965af31 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:39.579261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7283d531-7f7f-4d37-824f-ac5c5965af31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:39.695933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 709/1408] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:39.741715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:06:39.743280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:39.776862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:39.903525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b47e9-4a7f-4db0-baf2-61f233280ac9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:06:40.019058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:40.030253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:06:40.031656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:40.032003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:40.032275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:40.033881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:40.129835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:06:40.131455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7e803-1832-4519-8e89-8001ea5be9ec service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:06:40.150797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 700/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:06:49.691746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=51 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:50.148281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=49 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:57.497201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498f0e-0d3a-453b-abaf-0c186a06fbf0 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:57.497778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498f0e-0d3a-453b-abaf-0c186a06fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:57.497778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af498f0e-0d3a-453b-abaf-0c186a06fbf0 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:57.499135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 710/1410] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 10 11:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:57.504164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bddcd9-f6e6-4088-bf1f-ca76bfa6034c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:57.504604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bddcd9-f6e6-4088-bf1f-ca76bfa6034c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:57.504701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bddcd9-f6e6-4088-bf1f-ca76bfa6034c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:57.578641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-40bddcd9-f6e6-4088-bf1f-ca76bfa6034c None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:57.579848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-40bddcd9-f6e6-4088-bf1f-ca76bfa6034c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:57.683638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 701/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:06:57.698077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b291bd-fc40-4c66-b8dc-57bead014ed0 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:57.698438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b291bd-fc40-4c66-b8dc-57bead014ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:57.698744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b291bd-fc40-4c66-b8dc-57bead014ed0 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:57.700054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 711/1412] 10.222.0.49 () {54 vars in 918 bytes} [Wed Aug 10 11:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:06:57.704987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98624857-f05b-455c-8aca-e4b645a9d6ce None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:06:57.705267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98624857-f05b-455c-8aca-e4b645a9d6ce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:06:57.705537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98624857-f05b-455c-8aca-e4b645a9d6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:06:57.776325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:06:57.778944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-98624857-f05b-455c-8aca-e4b645a9d6ce None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:06:57.780697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-98624857-f05b-455c-8aca-e4b645a9d6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:06:57.894922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 702/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Aug 10 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:00.372081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.372603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.372603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.373966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:00.379949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167082-272c-478b-a1ab-698093f3f5e4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.380369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167082-272c-478b-a1ab-698093f3f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.380506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09167082-272c-478b-a1ab-698093f3f5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.447597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-09167082-272c-478b-a1ab-698093f3f5e4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:00.448813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-09167082-272c-478b-a1ab-698093f3f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:00.502472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:00.544097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:00.585547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:00.587070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:00.730709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61141666-8df6-4b7d-93ec-c65a1d419210 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:00.800877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:00.840755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:00.841355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.841639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.841913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.844136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:00.933172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:00.934809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2fba1-a320-4369-a198-0fdd3f63d239 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:00.952910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 713/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:01.837375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:01.837945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:01.837945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:01.839381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:01.844488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243849a-248b-47c4-8a44-12cc6da89d2a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:01.844890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243849a-248b-47c4-8a44-12cc6da89d2a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:01.845046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243849a-248b-47c4-8a44-12cc6da89d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:01.922166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c243849a-248b-47c4-8a44-12cc6da89d2a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:01.923379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c243849a-248b-47c4-8a44-12cc6da89d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:01.949957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:02.019446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:02.169896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:02.171356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:02.313058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a20d1a-81c4-4537-a941-530900344680 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:02.352110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:02.423217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:02.423741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:02.424018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:02.424288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:02.425866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:02.514376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:02.516837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e66a2b-4cfd-4fe5-8120-0de92ca86b23 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:02.538796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 705/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:03.436281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.436940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.436940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.438975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:03.443946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de168-6736-4965-87fd-e1858213d8aa None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.444407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de168-6736-4965-87fd-e1858213d8aa None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.444515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023de168-6736-4965-87fd-e1858213d8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.497556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:03.520497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-023de168-6736-4965-87fd-e1858213d8aa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:03.521710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-023de168-6736-4965-87fd-e1858213d8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:03.615145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:03.767991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.769498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:03.911471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640d32d-f5f7-4737-870c-a053586b233e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.924888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.023308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.023836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.024117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.024391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.025960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:04.113089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.114686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028705e-9cd5-453c-a308-dfaa2616e23d service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.132105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 716/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:04.986987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.987541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.987541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.988884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:04.993916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50571022-5431-48be-9caf-6c6bdbed237f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.994350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50571022-5431-48be-9caf-6c6bdbed237f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.994464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50571022-5431-48be-9caf-6c6bdbed237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:05.013878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:05.070846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-50571022-5431-48be-9caf-6c6bdbed237f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:05.072054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-50571022-5431-48be-9caf-6c6bdbed237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:05.166858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:05.317534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:05.319082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:05.424926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:05.459392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e85cb-e8c8-4fc5-91d3-7c647e580cbc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:05.569038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:05.569634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:05.569907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:05.570177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:05.571772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:05.645604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:05.659175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:05.660763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909130d1-4a5d-4faa-b974-d580204c068b service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:05.678437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 708/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:06.526449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:06.527019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:06.527019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:06.528311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:06.533766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bd534-405d-4899-a9a4-2532bdef222c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:06.534190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bd534-405d-4899-a9a4-2532bdef222c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:06.534302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403bd534-405d-4899-a9a4-2532bdef222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:06.608574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-403bd534-405d-4899-a9a4-2532bdef222c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:06.610358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-403bd534-405d-4899-a9a4-2532bdef222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:06.703721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:06.854978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:06.856404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:06.939148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:06.998855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e5a02-0bba-40a7-a6cf-dd878e854cad None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.107814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.108340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.108620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.108890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.110478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:07.157441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:07.197873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:07.199458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad605d-b95e-4925-805e-03c7039f8312 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:07.217289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 719/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:08.026250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.026815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.026940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.028202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:08.033125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f536fe-4dbe-48a5-a955-c735a51d4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.033693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f536fe-4dbe-48a5-a955-c735a51d4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.033693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f536fe-4dbe-48a5-a955-c735a51d4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.107888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-87f536fe-4dbe-48a5-a955-c735a51d4ff4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:08.109579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-87f536fe-4dbe-48a5-a955-c735a51d4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:08.209484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:08.357347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.358878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:08.406515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:08.520333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907be0fb-7cfa-4814-a028-7a742bca2dbe None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.637357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.637996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.638302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.638589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.640242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:08.646977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:08.728822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.730413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2aa29-2b16-4858-b3d7-2c0e16d29f2d service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.748474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 711/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:09.553562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:09.554091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:09.554091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:09.555400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:09.560556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea958-ae5a-46d5-b046-ba475b016f53 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:09.560986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea958-ae5a-46d5-b046-ba475b016f53 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:09.561194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ea958-ae5a-46d5-b046-ba475b016f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:09.637027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a01ea958-ae5a-46d5-b046-ba475b016f53 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:09.638227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a01ea958-ae5a-46d5-b046-ba475b016f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:09.704336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:09.732555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:09.886148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:09.887681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:09.904943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:10.030417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0484a-6000-4dc5-ba93-0018662ccc9f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:10.116064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:10.141394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:10.142233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:10.142648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:10.143018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:10.144750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:10.240785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:10.242380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238357f-fde1-4884-bc9d-7cc5589b7fd8 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:10.259912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 722/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:11.087548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.088121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.088121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.089470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:11.094846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1448-bd4f-4e7b-8a15-1efd43fdbbc0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.095263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1448-bd4f-4e7b-8a15-1efd43fdbbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.095397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c1448-bd4f-4e7b-8a15-1efd43fdbbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.173624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ee6c1448-bd4f-4e7b-8a15-1efd43fdbbc0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:11.174762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ee6c1448-bd4f-4e7b-8a15-1efd43fdbbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:11.215791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.269780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:11.421894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.423459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:11.568537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b698ce-d034-49cf-a6ff-01de68767c34 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.622641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.681030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.681652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.681930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.682201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.683835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:11.770811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.772319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c719a-6a06-4c46-bc7e-40776fe18b9f service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.789577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 714/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:12.590232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.590810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.590810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.592287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:12.597932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c451-b2af-4f03-9f14-71178e13d41a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.598396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c451-b2af-4f03-9f14-71178e13d41a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.598538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2c451-b2af-4f03-9f14-71178e13d41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.672955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-44e2c451-b2af-4f03-9f14-71178e13d41a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:12.674149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-44e2c451-b2af-4f03-9f14-71178e13d41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:12.676144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:12.768545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:12.915723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.917232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:13.059210 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9dd17f-3c4a-4c39-b20a-1562c045c0bc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:13.084532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:13.167908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:13.168692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:13.168971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:13.169237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:13.170830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:13.256220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:13.257783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2126c3-5c6b-404a-a16f-05e30d989e8d service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:13.276097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 725/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:15.193373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.193902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.193902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.195291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:15.200730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672bbd-9fac-4ae3-a5f2-3f8da5d0399b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.201157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672bbd-9fac-4ae3-a5f2-3f8da5d0399b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.201289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea672bbd-9fac-4ae3-a5f2-3f8da5d0399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.243936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.393757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ea672bbd-9fac-4ae3-a5f2-3f8da5d0399b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:15.394990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ea672bbd-9fac-4ae3-a5f2-3f8da5d0399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:15.490081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:07:15.529608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.531176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:15.632649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674008a0-64ea-4f2b-a843-efc5ebf9484c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:15.663356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.708549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:15.709148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.709436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.709710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.711335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:15.822130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:15.823928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c601b-e6e0-4a2d-840c-8ba49a504fe8 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:15.843096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 717/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:18.051230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:18.051789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:18.051789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:18.053107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:18.058301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bacb21-c468-4351-aa62-eaad72ef8424 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:18.058701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bacb21-c468-4351-aa62-eaad72ef8424 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:18.058909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bacb21-c468-4351-aa62-eaad72ef8424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:18.123456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-33bacb21-c468-4351-aa62-eaad72ef8424 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:18.124583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-33bacb21-c468-4351-aa62-eaad72ef8424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:18.136947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:18.223364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:18.262696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:18.264217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:18.373244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77dd585-1347-43cf-b583-fb087f487a0c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:18.387366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:18.451073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:18.451968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:18.451968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:18.452246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:18.454599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:18.563737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:18.565746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f014d3-0c43-408f-bc71-a29fec6ae0bb service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:18.593334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 728/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:20.940658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.941220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.941220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.942533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:20.947796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98477-5e5c-45a1-928f-2c80dd0456fa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.948252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98477-5e5c-45a1-928f-2c80dd0456fa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.948357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d98477-5e5c-45a1-928f-2c80dd0456fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.994977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:21.013401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a7d98477-5e5c-45a1-928f-2c80dd0456fa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:21.014640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a7d98477-5e5c-45a1-928f-2c80dd0456fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:21.106706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:21.146677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:21.148227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:21.248642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e43aaa-4d39-4da3-a4c5-5745c925ef5e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:21.288236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:21.322688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:21.323298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:21.323577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:21.323844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:21.325456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:21.438685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:21.440040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161abee-53cf-4da5-a95c-15f00da95c3d service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:21.459855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 720/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:23.561847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.562411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.562411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.564284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:23.569778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447b5f8-b15e-4a8d-aae0-20fdd467a667 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.570172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447b5f8-b15e-4a8d-aae0-20fdd467a667 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.570327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447b5f8-b15e-4a8d-aae0-20fdd467a667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.636182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0447b5f8-b15e-4a8d-aae0-20fdd467a667 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:23.637378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0447b5f8-b15e-4a8d-aae0-20fdd467a667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:23.658002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:23.733670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:23.773522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.775059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:23.875877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23b323f-4612-4653-a212-be500d4436d1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:23.896401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:23.950279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:23.950909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.951194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.951465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.953064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:24.059208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.060660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56bd6e-2e0f-4cdb-abf3-92f53a7233c9 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.078284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 731/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:26.121951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:26.122420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:26.122420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:26.123799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:26.129524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c3b8-6f33-4919-a05e-1c4a13208b3c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:26.129899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c3b8-6f33-4919-a05e-1c4a13208b3c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:26.130073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137c3b8-6f33-4919-a05e-1c4a13208b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:26.192578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:26.195389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b137c3b8-6f33-4919-a05e-1c4a13208b3c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:26.197182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b137c3b8-6f33-4919-a05e-1c4a13208b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:26.290990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:26.329168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:26.330720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:26.430068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f303ca-40d3-407f-b3e9-f29a30f912c5 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:26.477630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:26.502226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:26.502868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:26.503158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:26.503428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:26.505019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:26.613694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:26.615179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd76bef1-efcd-4cdd-8774-ce885104f21c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:26.633032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 723/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:28.685117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:28.685640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:28.685640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:28.686948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:28.692192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56abbc4-1c97-4927-b8fa-776c863b0364 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:28.692585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56abbc4-1c97-4927-b8fa-776c863b0364 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:28.692748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56abbc4-1c97-4927-b8fa-776c863b0364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:28.758391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f56abbc4-1c97-4927-b8fa-776c863b0364 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:28.759820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f56abbc4-1c97-4927-b8fa-776c863b0364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:28.789425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:28.857320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:28.895058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:28.896635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:28.996704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91135614-e45e-40a9-9a3f-37a49775cd28 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:29.023436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:29.068822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:29.069416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:29.069693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:29.069963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:29.071601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:29.177558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:29.178999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031ebb0-6171-4a0c-bcf6-eaa9c06d6898 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:29.197077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 734/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:31.256367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:31.256927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:31.256927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:31.258262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:31.263432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58ed1-f163-441c-8919-684b310aedfb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:31.263859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58ed1-f163-441c-8919-684b310aedfb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:31.263971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db58ed1-f163-441c-8919-684b310aedfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:31.330535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8db58ed1-f163-441c-8919-684b310aedfb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:31.331771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8db58ed1-f163-441c-8919-684b310aedfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:31.339835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:31.429564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:31.466503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:31.468114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:31.569970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814aa55-41ce-471b-a573-66ea238285a9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:31.623623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:31.641817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:31.642414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:31.642725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:31.643022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:31.644612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:31.749590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:31.751099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf60892-67a9-4fa9-beaa-48f4f24221ab service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:31.769796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 726/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:33.802270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:33.802834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:33.802834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:33.804196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:33.809346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95837d8b-732d-472f-8ec4-46c46787dfb8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:33.809739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95837d8b-732d-472f-8ec4-46c46787dfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:33.809896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95837d8b-732d-472f-8ec4-46c46787dfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:33.874310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-95837d8b-732d-472f-8ec4-46c46787dfb8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:33.875579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95837d8b-732d-472f-8ec4-46c46787dfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:33.913565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:33.970774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:34.008185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:34.009748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:34.111421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1621a-5fee-438f-aafd-3052cebefbe4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:34.149440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:34.185347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:34.185939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:34.186215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:34.186483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:34.188844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:34.293847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:34.295534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ca459-4dbb-4803-ae46-90a2333d514e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:34.314083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 737/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:36.323612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:36.324156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:36.324275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:36.325518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:36.330926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14d73-2743-4e73-a387-d95ca036aaeb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:36.331351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14d73-2743-4e73-a387-d95ca036aaeb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:36.331485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b14d73-2743-4e73-a387-d95ca036aaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:36.395668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b4b14d73-2743-4e73-a387-d95ca036aaeb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:36.396856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b4b14d73-2743-4e73-a387-d95ca036aaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:36.414489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:36.490276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:36.526240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:36.527806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:36.630282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73aa8-ff50-48ea-987d-ddfceefc606f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:36.694533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:36.701686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:36.702281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:36.702559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:36.702885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:36.704952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:36.809655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:36.811040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae24948-6337-476e-bffd-591219837e2c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:36.829393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 729/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:38.805818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:38.806385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:38.806385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:38.807763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:38.813279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d46b-bed8-43c0-8006-aeb817af02bf None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:38.813715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d46b-bed8-43c0-8006-aeb817af02bf None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:38.813821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d46b-bed8-43c0-8006-aeb817af02bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:38.880759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f622d46b-bed8-43c0-8006-aeb817af02bf None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:38.881977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f622d46b-bed8-43c0-8006-aeb817af02bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:38.928828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:38.976604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:39.014307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:39.015893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:39.119317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc102f-9830-4faf-996b-f7b2142509c4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:39.166059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:39.192529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:39.193148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:39.193445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:39.193714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:39.195304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:39.300114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:39.301521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc39db74-bf96-4c5f-9713-856d23ec30fe service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:39.319490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 740/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:07:48.974828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=22 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:49.317197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=66 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:58.670548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6727d-5b0a-4600-9aa6-363be3b9252d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:58.671106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6727d-5b0a-4600-9aa6-363be3b9252d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:58.671106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6727d-5b0a-4600-9aa6-363be3b9252d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:58.672878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 731/1471] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:58.678103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f4e7-0640-415f-bb8d-04d5f59ea429 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:58.678536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f4e7-0640-415f-bb8d-04d5f59ea429 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:58.678713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37f4e7-0640-415f-bb8d-04d5f59ea429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:58.744027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7e37f4e7-0640-415f-bb8d-04d5f59ea429 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:58.745213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7e37f4e7-0640-415f-bb8d-04d5f59ea429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:58.839205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 741/1472] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:58.921885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:58.922798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:58.923102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:58.924426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 732/1473] 10.222.0.49 () {54 vars in 967 bytes} [Wed Aug 10 11:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:58.929076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692f62-8a0b-4836-9c77-4be087d0c7e9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:58.929364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692f62-8a0b-4836-9c77-4be087d0c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:58.929641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e692f62-8a0b-4836-9c77-4be087d0c7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:59.002765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9e692f62-8a0b-4836-9c77-4be087d0c7e9 None None] MFA Rules not processed for user `eacd5dcb5de7474da98c2e7f30a4e6db`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:59.003961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9e692f62-8a0b-4836-9c77-4be087d0c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:59.024160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:59.079426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 742/1474] 10.222.0.49 () {58 vars in 1064 bytes} [Wed Aug 10 11:07:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:07:59.088732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:59.090390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:59.195891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cb1116-1422-40c3-976c-c912aa680e47 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:59.266705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:59.267302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:59.267585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:59.267856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:59.269439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:59.321390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:59.376414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:59.378603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384fddf-f546-483b-a2ea-72d04708f942 service glance] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:59.397817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 733/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:08:01.464508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:01.465078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:01.465078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:01.466408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 743/1476] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:01.472244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb177d7e-20de-49b8-b619-4d0e2f079cf6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:01.472656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb177d7e-20de-49b8-b619-4d0e2f079cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:01.472801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb177d7e-20de-49b8-b619-4d0e2f079cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:01.540277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-fb177d7e-20de-49b8-b619-4d0e2f079cf6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:01.541500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fb177d7e-20de-49b8-b619-4d0e2f079cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:01.636042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 734/1477] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:08:01.659798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:01.661303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:01.786055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3345ef9-2d07-4096-bd8f-7607bced4de6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:01.835005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:01.880027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:01.880629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:01.880911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:01.881188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:01.881829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:01.883573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:08:01.883573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:08:01.883573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:08:01.884701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc46053-9b79-4df6-8af2-484def6a7baa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:01.903544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 744/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Wed Aug 10 11:08:01 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:08:03.904433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:03.904888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:03.905001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:03.906411 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 735/1479] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:03.911645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6819-5210-4760-81e4-779a96b4a1a5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:03.912062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6819-5210-4760-81e4-779a96b4a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:03.912251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f6819-5210-4760-81e4-779a96b4a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:03.976707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-243f6819-5210-4760-81e4-779a96b4a1a5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:03.977899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-243f6819-5210-4760-81e4-779a96b4a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:04.072131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 745/1480] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:08:04.114809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:04.116355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:04.126001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:04.257834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4f9b-39ef-42dc-bcc8-c04a48a1f809 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:04.337141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:04.369849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:04.370385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:04.370748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:04.371021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:04.372613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:04.461577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:04.463225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c55611f-00c5-497c-8fbf-07e61166c90b service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:04.481442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 736/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:08:06.653813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:06.654379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:06.654379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:06.655700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 746/1482] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:06.660801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b9ff-a01c-47ec-b9e9-20b05bee4eff None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:06.661207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b9ff-a01c-47ec-b9e9-20b05bee4eff None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:06.661347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28b9ff-a01c-47ec-b9e9-20b05bee4eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:06.725827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ce28b9ff-a01c-47ec-b9e9-20b05bee4eff None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:06.727056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ce28b9ff-a01c-47ec-b9e9-20b05bee4eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:06.773663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:06.824734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 737/1483] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:08:06.860650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:06.862251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:06.885589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:06.964361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53efd8c-fd5b-44b5-af85-e7b3dd72da57 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:07.036688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:07.037287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:07.037566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:07.037837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:07.039464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:07.113069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:07.146833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:07.148250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0291b-b537-405b-be7d-eaaa34a82435 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:07.166131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 747/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:08:09.198421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:09.198996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:09.198996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:09.200359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 738/1485] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:09.206023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8cd7-53ec-4ead-a58f-0f01a5b65ee3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:09.206440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8cd7-53ec-4ead-a58f-0f01a5b65ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:09.206592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c8cd7-53ec-4ead-a58f-0f01a5b65ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:09.270270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-493c8cd7-53ec-4ead-a58f-0f01a5b65ee3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:09.271501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-493c8cd7-53ec-4ead-a58f-0f01a5b65ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:09.364549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 748/1486] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:08:09.403177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:09.404705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:09.505830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215778f0-ba03-40cf-b502-27615adcf24a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:09.575971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:09.577747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:09.579026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:09.579499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:09.579815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:09.581747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:09.687429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:09.689427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0eea-a78f-4b6b-8dbb-251166ec25aa service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:09.707258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 739/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:08:11.902532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:11.903076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:11.903191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:11.904563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 749/1488] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:11.910522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb87e-655f-4293-856d-4d39fe5d2331 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:11.910954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb87e-655f-4293-856d-4d39fe5d2331 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:11.911132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeb87e-655f-4293-856d-4d39fe5d2331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:11.977286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9daeb87e-655f-4293-856d-4d39fe5d2331 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:11.978521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9daeb87e-655f-4293-856d-4d39fe5d2331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:12.031249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:12.077354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 740/1489] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:08:12.118528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:12.120058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:12.156213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:12.224640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c10bdd-9784-44c6-8ded-4b27dcb363fe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:12.297143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:12.297842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:12.298174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:12.298494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:12.300375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:12.383834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:12.413152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:12.415041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597164c-1422-4652-b66f-97731413f11b service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:12.432894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 750/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:08:14.791257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:14.791831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:14.791831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:14.793179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 741/1491] 10.222.0.49 () {54 vars in 927 bytes} [Wed Aug 10 11:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:08:14.799911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd930-0205-463e-9116-406def9f668f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:14.800409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd930-0205-463e-9116-406def9f668f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:14.800511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bd930-0205-463e-9116-406def9f668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:14.865919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-396bd930-0205-463e-9116-406def9f668f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:08:14.866673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-396bd930-0205-463e-9116-406def9f668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:14.962125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 751/1492] 10.222.0.49 () {58 vars in 1024 bytes} [Wed Aug 10 11:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:08:15.001311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:08:15.002863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:15.107048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac2d9b7-978f-4469-b250-047f1001b9df None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:08:15.178078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:08:15.178625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:08:15.178902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:08:15.179167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:08:15.180729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:08:15.190077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:15.285342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:08:15.286789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8120-a458-47ed-bbc9-15d30fcdd93c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:08:15.304821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 742/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:08:24.960316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=82 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:08:25.302627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=50 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:24.612514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:24.613070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:24.613070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:24.685538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:09:24.686762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:24.778892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 752/1494] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:09:24.786945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:24.787453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:24.787570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:24.862718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:09:24.863837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:24.956719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 743/1495] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:09:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:09:24.961330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:24.962516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:24.974095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace04ad9-8c6a-4c59-b4c8-0a79fefb8fb0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.032821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:25.067163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.067758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.068033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.068301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.068937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:09:25.069234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.070228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.070228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:09:25.070228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:09:25.071126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.081902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 753/1496] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:09:25.090143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:25.099578 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908926a-43dd-4a2b-84b0-9ca505edf82b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.159284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:25.192132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.192723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.192999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.193266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.193875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.195606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.195606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:09:25.195606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:09:25.196799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.249455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 744/1497] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:09:25 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:09:25.253806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:25.255375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:25.287681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e16a5b-b552-4bd7-b285-278feee4dd09 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.378681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.379270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.379550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.379818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.380446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.381433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.381433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:09:25.381433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:09:25.382686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.424026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 754/1498] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:09:25 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:09:25.428033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:25.429583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:25.489072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468cd15-ac2b-482a-9c82-e9670612ab37 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.569142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:25.580920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.581523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.581808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.582079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.582755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.583766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.583766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:09:25.583766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:09:25.584650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.597372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 745/1499] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:09:25.601440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:25.602960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:25.614205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c344fa1-8b6c-49ff-9cf9-86958da30d5b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.638454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:25.708471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.709070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.709348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.709618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.710253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.711290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.711290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:09:25.711290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:09:25.712162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da498c86-ed61-4919-a0be-345812642ff1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.724998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 755/1500] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:09:25.729128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:25.730702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:25.763825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ed796-5739-4765-93fb-94018e914d5b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:25.859515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:25.860813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.861093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.861363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] PATH_INFO: `/v3/projects/97cd1208d0e94b59b59b5ff52ee18a1c/users/5a7df9970ec44f409b5e43aa88f88cd8/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.884754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cd1208d0e94b59b59b5ff52ee18a1c, user_id=5a7df9970ec44f409b5e43aa88f88cd8, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:25.885791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:25.885791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:09:25.885791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:09:25.887770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec071c-360e-48da-9d2c-75582f9e2c2f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:25.911776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 746/1501] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:09:25 2022] PUT /identity/v3/projects/97cd1208d0e94b59b59b5ff52ee18a1c/users/5a7df9970ec44f409b5e43aa88f88cd8/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:09:25.924324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2e23-2d4b-48c4-a9c2-17114510a7ec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:25.924613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2e23-2d4b-48c4-a9c2-17114510a7ec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:25.924892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c2e23-2d4b-48c4-a9c2-17114510a7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:25.980110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-501c2e23-2d4b-48c4-a9c2-17114510a7ec None None] MFA Rules not processed for user `5a7df9970ec44f409b5e43aa88f88cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:09:25.981244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-501c2e23-2d4b-48c4-a9c2-17114510a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:26.056928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 756/1502] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:09:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:09:26.070127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.070424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.070741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.072081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 747/1503] 10.222.0.49 () {54 vars in 964 bytes} [Wed Aug 10 11:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:09:26.076845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9f51-824f-4bac-a73b-0bb4316b4322 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.077133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9f51-824f-4bac-a73b-0bb4316b4322 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.077412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d9f51-824f-4bac-a73b-0bb4316b4322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.108409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:26.153063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f11d9f51-824f-4bac-a73b-0bb4316b4322 None None] MFA Rules not processed for user `5882280742154859b52fa47c350a137f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:09:26.154266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f11d9f51-824f-4bac-a73b-0bb4316b4322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:26.228128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 757/1504] 10.222.0.49 () {58 vars in 1061 bytes} [Wed Aug 10 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:09:26.236015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:26.237511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:26.287217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:26.339945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6f1ae-460e-41ba-bdee-abdc5325266b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:26.410745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:26.411341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.411620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.411892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.413482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:26.508218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:26.509576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:26.512984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:26.527999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 748/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:09:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:09:26.574237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec81bf-b274-4c80-a636-dbf74c254d34 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.574525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec81bf-b274-4c80-a636-dbf74c254d34 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.574834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec81bf-b274-4c80-a636-dbf74c254d34 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.576095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 758/1506] 10.222.0.49 () {52 vars in 830 bytes} [Wed Aug 10 11:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:09:26.580218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:26.581427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:26.677961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d6a8c-4d26-476c-bcbe-858341da9ebc service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:26.769658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:26.770214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.770495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.770876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] PATH_INFO: `/v3/users/5a7df9970ec44f409b5e43aa88f88cd8` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.782893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7df9970ec44f409b5e43aa88f88cd8)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:26.783918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:26.783918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:09:26.783918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:09:26.785132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:26.794851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:26.852977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7df9970ec44f409b5e43aa88f88cd8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:26.869130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7df9970ec44f409b5e43aa88f88cd8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:26.878983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7df9970ec44f409b5e43aa88f88cd8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:26.879408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7df9970ec44f409b5e43aa88f88cd8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:26.879908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7df9970ec44f409b5e43aa88f88cd8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:26.907308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 749/1507] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:09:26 2022] DELETE /identity/v3/users/5a7df9970ec44f409b5e43aa88f88cd8 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:09:26.911587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:26.912030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:26.912156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:26.967344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:09:26.968542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:27.041939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:27.055844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 759/1508] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:09:27.066687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:27.067900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:27.166657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec8af0-2efa-4d5e-a75b-0ce6549a684a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:27.220720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:27.241472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:27.242073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:27.242350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:27.242652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:27.244233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:27.347765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:27.349118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8506eb6d-49e1-4b8c-a270-a12dadc36461 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:27.367254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 750/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:09:27.694504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:09:27.695747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:09:27.707449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cccae-8054-4d78-ad6e-2c52bed27cde None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:09:27.804623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:09:27.805252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:09:27.805530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:09:27.805802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] PATH_INFO: `/v3/projects/97cd1208d0e94b59b59b5ff52ee18a1c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:09:27.815512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cd1208d0e94b59b59b5ff52ee18a1c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:09:27.816527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:09:27.816527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:09:27.816527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:09:27.817717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:09:27.882488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cd1208d0e94b59b59b5ff52ee18a1c', 'request_id': 'req-225d627e-01ec-4441-a7d2-2596dec86fc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:27.888360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cd1208d0e94b59b59b5ff52ee18a1c', 'request_id': 'req-225d627e-01ec-4441-a7d2-2596dec86fc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:27.897979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cd1208d0e94b59b59b5ff52ee18a1c', 'request_id': 'req-225d627e-01ec-4441-a7d2-2596dec86fc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:27.928751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] The token cache is being invalidate because project 97cd1208d0e94b59b59b5ff52ee18a1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:09:27.929397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:27.929552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-225d627e-01ec-4441-a7d2-2596dec86fc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:09:27.930925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 760/1510] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:09:27 2022] DELETE /identity/v3/projects/97cd1208d0e94b59b59b5ff52ee18a1c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:09:37.365044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=62 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:09:37.926819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:10:55.710559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:55.711187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:55.711187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:55.712778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:10:55.717632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067483-1214-403b-ba03-8e7c3a8dd2aa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:55.718106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067483-1214-403b-ba03-8e7c3a8dd2aa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:55.718223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2067483-1214-403b-ba03-8e7c3a8dd2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:55.797206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f2067483-1214-403b-ba03-8e7c3a8dd2aa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:10:55.798414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f2067483-1214-403b-ba03-8e7c3a8dd2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:10:55.901639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:10:56.054501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:10:56.057084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:10:56.170461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345888e-6fef-4699-a8c9-1d7302e80561 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:10:56.179521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:10:56.181071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:56.181561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:56.182026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:56.190430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:10:56.192078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a64f20-550a-4758-bb82-ae2649af02dc service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:10:56.201925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 752/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:10:57.025459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae6509-c897-4fc7-8ecb-29b9a6705f2c None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:57.025982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae6509-c897-4fc7-8ecb-29b9a6705f2c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:57.025982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae6509-c897-4fc7-8ecb-29b9a6705f2c None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:57.027405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:10:57.032205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f21a-293d-43d8-af0c-7a9653ce8411 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:57.032603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f21a-293d-43d8-af0c-7a9653ce8411 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:57.032788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91f21a-293d-43d8-af0c-7a9653ce8411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:57.051816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3f91f21a-293d-43d8-af0c-7a9653ce8411 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:10:57.053007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3f91f21a-293d-43d8-af0c-7a9653ce8411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:10:57.061546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:10:57.209686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae6509-c897-4fc7-8ecb-29b9a6705f2c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:10:57.223510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae6509-c897-4fc7-8ecb-29b9a6705f2c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:10:57.229388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:10:57.229967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:57.230242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:57.230511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:57.238350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:10:57.239933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e68e3aa-c96a-449b-9769-c46cba87c9b9 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:10:57.249578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 763/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:10:57.615789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b03a9-ee9c-4983-9cd1-5d5a49c2d8b2 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:57.615981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b03a9-ee9c-4983-9cd1-5d5a49c2d8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:57.616260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b03a9-ee9c-4983-9cd1-5d5a49c2d8b2 None None] PATH_INFO: `/v3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:57.617575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 754/1517] 10.222.0.52 () {54 vars in 925 bytes} [Wed Aug 10 11:10:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:10:57.624533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704929-2f80-45e6-9d34-66181bd59bf0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:10:57.624817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704929-2f80-45e6-9d34-66181bd59bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:10:57.625088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc704929-2f80-45e6-9d34-66181bd59bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:10:57.682818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bc704929-2f80-45e6-9d34-66181bd59bf0 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:10:57.684184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc704929-2f80-45e6-9d34-66181bd59bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:10:57.700803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:10:57.753158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 764/1518] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 10 11:10:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:11:06.715464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e034c-bf92-43c6-9a31-d8a027e86315 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:06.716005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e034c-bf92-43c6-9a31-d8a027e86315 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:06.716005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e034c-bf92-43c6-9a31-d8a027e86315 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:06.717346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 755/1519] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 10 11:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:06.722125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd156dc7-e653-4234-bc24-a57c3cb23242 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:06.722535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd156dc7-e653-4234-bc24-a57c3cb23242 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:06.722709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd156dc7-e653-4234-bc24-a57c3cb23242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:06.741478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bd156dc7-e653-4234-bc24-a57c3cb23242 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:06.742672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bd156dc7-e653-4234-bc24-a57c3cb23242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:06.751669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 765/1520] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 10 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:06.759256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e034c-bf92-43c6-9a31-d8a027e86315 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:06.772167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e034c-bf92-43c6-9a31-d8a027e86315 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:06.777015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:06.777589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:06.777866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:06.778137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:06.785964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:06.787550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3fd860-8f8f-49ff-9dff-b7b1173d7f4f service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:06.796762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 756/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:08.473623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6b1b-be1f-4358-b6fd-28987d5b38cd None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:08.474169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6b1b-be1f-4358-b6fd-28987d5b38cd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:08.474169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c6b1b-be1f-4358-b6fd-28987d5b38cd None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:08.475533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:08.480742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85939cf-b3ea-41fc-a1aa-c82c81a45a95 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:08.481161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85939cf-b3ea-41fc-a1aa-c82c81a45a95 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:08.481295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85939cf-b3ea-41fc-a1aa-c82c81a45a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:08.502284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b85939cf-b3ea-41fc-a1aa-c82c81a45a95 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:08.503515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b85939cf-b3ea-41fc-a1aa-c82c81a45a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:08.511159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:08.661237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6b1b-be1f-4358-b6fd-28987d5b38cd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:08.675983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c6b1b-be1f-4358-b6fd-28987d5b38cd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:08.682110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:08.682727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:08.683007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:08.683279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:08.691163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:08.692749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833226c1-7017-43cb-a08a-90b5db74acac service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:08.702167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 767/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:09.515777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028c2d-91c4-4e46-8a8d-4bb76d73ec20 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:09.516454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028c2d-91c4-4e46-8a8d-4bb76d73ec20 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:09.516454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94028c2d-91c4-4e46-8a8d-4bb76d73ec20 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:09.517665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:09.522318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9f93-db5a-40a1-9013-6ea363f57b03 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:09.522745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9f93-db5a-40a1-9013-6ea363f57b03 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:09.523001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a9f93-db5a-40a1-9013-6ea363f57b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:09.542946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-856a9f93-db5a-40a1-9013-6ea363f57b03 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:09.544133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-856a9f93-db5a-40a1-9013-6ea363f57b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:09.551904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:09.702267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94028c2d-91c4-4e46-8a8d-4bb76d73ec20 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:09.716876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94028c2d-91c4-4e46-8a8d-4bb76d73ec20 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:09.722334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:09.722955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:09.723237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:09.723511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:09.732047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:09.733624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8c5f09-05f4-4aef-bcb3-432b39f3c44e service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:09.742878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 759/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:13.656695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:13.657224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:13.657224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:13.658492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:13.663588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c877-38c3-4f48-a5ae-b6ab5f61ffce None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:13.663969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c877-38c3-4f48-a5ae-b6ab5f61ffce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:13.664130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56c877-38c3-4f48-a5ae-b6ab5f61ffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:13.683220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0a56c877-38c3-4f48-a5ae-b6ab5f61ffce None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:13.684402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0a56c877-38c3-4f48-a5ae-b6ab5f61ffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:13.692108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:13.768119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:13.770373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:13.850296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e15f5a-ea9d-41d5-841b-423aaeafca33 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:13.854321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:13.854991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:13.855274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:13.855542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:13.865163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:13.866602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7589d-3a59-4513-b2fd-aa39ac479fe4 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:13.878834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 770/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:15.873158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:15.873696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:15.873696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:15.875092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:15.880212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41962d8-0f8a-49fc-a69d-400b4a97041c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:15.880643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41962d8-0f8a-49fc-a69d-400b4a97041c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:15.880750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41962d8-0f8a-49fc-a69d-400b4a97041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:15.900849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a41962d8-0f8a-49fc-a69d-400b4a97041c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:15.902042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a41962d8-0f8a-49fc-a69d-400b4a97041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:15.909837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:15.946457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:15.948160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:15.960776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd091ae9-4e70-4efa-a6c6-471421d4f6bb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:15.964244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:15.964822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:15.965097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:15.965366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:15.973942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:15.975376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6ab06-81ed-4394-97de-cb666b6cf12d service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:15.985065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 762/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:19.611534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:19.612069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:19.612069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:19.613471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:19.619240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d1029-2406-4f20-bbe9-78c2b8f7253b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:19.619641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d1029-2406-4f20-bbe9-78c2b8f7253b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:19.619787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d1029-2406-4f20-bbe9-78c2b8f7253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:19.627248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=81 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:11:19.639931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-853d1029-2406-4f20-bbe9-78c2b8f7253b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:19.641127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-853d1029-2406-4f20-bbe9-78c2b8f7253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:19.648966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:19.949640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:19.951307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:20.013738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:11:20.027999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2386a-0e8f-4682-b0d0-0e029c9c5d13 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:20.031573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:20.032167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:20.032453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:20.032730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:20.042098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:20.043590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b4c6b-c084-47ec-a338-353dfc9528c8 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:20.053062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 773/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:22.056843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:22.057371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:22.057371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:22.058785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:11:22.063818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e03-a50b-4eb3-8c37-775943a96239 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:22.064254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e03-a50b-4eb3-8c37-775943a96239 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:22.064356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40e03-a50b-4eb3-8c37-775943a96239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:22.083684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9ca40e03-a50b-4eb3-8c37-775943a96239 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:11:22.084866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9ca40e03-a50b-4eb3-8c37-775943a96239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:22.093100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:11:22.397069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:11:22.398875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:11:22.411647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77829642-b797-4fe1-8c69-e9ecf15f1812 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:11:22.415136 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:11:22.415726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:11:22.416009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:11:22.416284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:11:22.424935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:11:22.426355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a1c5-dbdf-4515-a66a-270934dd443b service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:11:22.436195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 765/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:11:32.079009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=11 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:11:32.432215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=8 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:46.323242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320395-d5da-4d5b-9c04-6bb32397d458 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.323898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320395-d5da-4d5b-9c04-6bb32397d458 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.323898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83320395-d5da-4d5b-9c04-6bb32397d458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.353299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-83320395-d5da-4d5b-9c04-6bb32397d458 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.354563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-83320395-d5da-4d5b-9c04-6bb32397d458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.357942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d333-6ed2-4879-89bd-ec9375c7e2a2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.358895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d333-6ed2-4879-89bd-ec9375c7e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.359426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10d333-6ed2-4879-89bd-ec9375c7e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.374623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 775/1540] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:46.378228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.378720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.378994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.387852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-7c10d333-6ed2-4879-89bd-ec9375c7e2a2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.392665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7c10d333-6ed2-4879-89bd-ec9375c7e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.406460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 766/1541] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:12:46.416846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.418095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.428206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.429116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.429609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.436142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 776/1542] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:12:46.437646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.456492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3aba29-5117-4761-a05b-6c5ca3c6181b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.462237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.463760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.473360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.473988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.474273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.474546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.475241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:46.475545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.477428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.480663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 767/1543] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:46.484880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.495248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb55624-0ab8-4489-859d-607c7f88a682 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.499642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 777/1544] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:46.508223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.508594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.509470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.510499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.511013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.511875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:46.512514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.513745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:46.513745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:12:46.513745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:12:46.515048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.526603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c25421-ea93-476a-a121-b72b6d5f79c8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.533880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 768/1545] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:12:46.539152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.539794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.540078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.540352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.541607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.542657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:46.542657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:12:46.542657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:12:46.543885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.551810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.568456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da54da22-6a45-40c7-bd57-578a0f94407d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.589483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.590111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.590393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.598730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.600513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.606156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.652470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 778/1546] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/projects => generated 391 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:12:46.656365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.658863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 769/1547] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:46.663744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.771740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7042ecc-697b-4d9f-a2e4-502be6930322 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.776615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23b503-567d-4c39-94de-bca913bc1ee3 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.781013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.781653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.781934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.782208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.782973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.784914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66706a-7205-4651-869d-f0420b52a612 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.793839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.794931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.795447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.795941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.796821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.798089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:46.798089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:12:46.798089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:12:46.799923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.825832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 779/1548] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:12:46.827838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.828421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.828704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.842229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 770/1549] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:12:46.843397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.870268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.871517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb216af-e5b5-4d85-98a9-1fe3675970e7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.871639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.890568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 780/1550] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:46.891509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.893254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.894164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.894707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.895187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.896079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.897904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44167d53-6413-48f1-92b8-f46e0d3f87c9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.905885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada0a46-3104-4ccc-ad45-6a28c571016c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:46.918471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 771/1551] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:46.920399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:46.921032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.922019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1846-869a-4c74-a50e-3619fc9bde0d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.922299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1846-869a-4c74-a50e-3619fc9bde0d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.922566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c1846-869a-4c74-a50e-3619fc9bde0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.923350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.923643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.924457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:46.929215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965ae6c1-589b-4faf-8e8c-70cb806bf0ae admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:46.946333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 781/1552] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:46.948172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.948831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.949335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.958211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e40c1846-869a-4c74-a50e-3619fc9bde0d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.960277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e40c1846-869a-4c74-a50e-3619fc9bde0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.971881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 772/1553] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:46.972196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:46.973169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:46.973994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300f4f-04cd-4aca-8866-43305ac3212a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:46.974500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300f4f-04cd-4aca-8866-43305ac3212a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:46.975019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300f4f-04cd-4aca-8866-43305ac3212a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:46.982800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 782/1554] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:46.983663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:46.999308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4f300f4f-04cd-4aca-8866-43305ac3212a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.000889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4f300f4f-04cd-4aca-8866-43305ac3212a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.001330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c01a7-808e-4260-8ad3-76eba7b8f7a5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.008536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.009182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.009519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.009800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.010449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.012049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 773/1555] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.012292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.014124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2485ecc-0ccd-4c51-abe8-e536a7720883 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.015126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2485ecc-0ccd-4c51-abe8-e536a7720883 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.015965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2485ecc-0ccd-4c51-abe8-e536a7720883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.023090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 783/1556] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:47.023752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.033938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ade16a-84de-4ca1-9a8b-b06f0f979896 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.038732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.039320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.039601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.039877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.040535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.040824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c2485ecc-0ccd-4c51-abe8-e536a7720883 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.042140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c2485ecc-0ccd-4c51-abe8-e536a7720883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.042290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417c3b7a-b038-44a4-878d-74c3890c0f53 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.055833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 774/1557] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.055833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12001-0926-4434-a59e-1cc379246278 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.055833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12001-0926-4434-a59e-1cc379246278 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.055833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e12001-0926-4434-a59e-1cc379246278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.057215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 784/1558] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:47.058875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.058875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.058875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.075814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c8e12001-0926-4434-a59e-1cc379246278 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.076997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c8e12001-0926-4434-a59e-1cc379246278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.079088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.080259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.085054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 775/1559] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.086025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ca509-a103-430a-9df4-953de454768f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.086305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ca509-a103-430a-9df4-953de454768f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.086606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ca509-a103-430a-9df4-953de454768f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.088331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 785/1560] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.089081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.098707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7122d-e541-472b-92f4-9e09e2f41363 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.103553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.104197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.104475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.104747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.105394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.105695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.107329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.112711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-792ca509-a103-430a-9df4-953de454768f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.113901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-792ca509-a103-430a-9df4-953de454768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.119609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 786/1561] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.120424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.124093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 776/1562] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.131581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ca509-a103-430a-9df4-953de454768f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.132128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987888fd-986a-402e-ac38-b0eaafa1aa25 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.137413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.138060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.138343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.138647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] PATH_INFO: `/v3/projects/429a80ecb8464ef3ad67a23a06fc10f7/users/5d1368fb915444b599b3fcf3f5586789/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.141433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ca509-a103-430a-9df4-953de454768f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.145996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.146667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.146955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.147221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] PATH_INFO: `/v3/projects/a4b7e68000ed472c88481b04ad050898/users/e01531151f024622b7dae0dcb23aa8f4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.153072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429a80ecb8464ef3ad67a23a06fc10f7, user_id=5d1368fb915444b599b3fcf3f5586789, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.154086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:47.154086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:12:47.154086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:12:47.156045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69abb5-a96d-4725-9596-304144e89190 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.162041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4b7e68000ed472c88481b04ad050898, user_id=e01531151f024622b7dae0dcb23aa8f4, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.164799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.169613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 787/1563] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:47 2022] PUT /identity/v3/projects/429a80ecb8464ef3ad67a23a06fc10f7/users/5d1368fb915444b599b3fcf3f5586789/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:47.171036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.171324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.171594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.178019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 777/1564] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:47 2022] PUT /identity/v3/projects/a4b7e68000ed472c88481b04ad050898/users/e01531151f024622b7dae0dcb23aa8f4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:47.179099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:47.179243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.201446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.202686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.224628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:47.253266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-f657-4fdc-bf73-a1e1828b309c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.257905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.258509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.258818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.259092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.260359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.260662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.262268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b8e91-58a1-41f4-9529-bb66a9b27ec6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.272256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 788/1565] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.272256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.275636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 778/1566] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.276802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.277095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.277374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.288487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01a2dc-a0f0-47ca-b988-da378b7dc534 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.293966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.294731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.295011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.295284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.295940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.296243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.297841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9921f5b-1028-4bdf-bf23-221fbc54c0e2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.298778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.299974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.309169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 779/1567] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.311359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.314871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 789/1568] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.316393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b9d92-f9fa-46b2-bdea-170bfaf0b0a5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.316922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b9d92-f9fa-46b2-bdea-170bfaf0b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.317195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b9d92-f9fa-46b2-bdea-170bfaf0b0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.321467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b175247-631d-4336-80ae-08e5255a8641 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.326327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.326976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.327257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.327530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.328155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.330056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f905e1-7a89-4969-94c6-697c17f41ee9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.339188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fc5b9d92-f9fa-46b2-bdea-170bfaf0b0a5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.340777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc5b9d92-f9fa-46b2-bdea-170bfaf0b0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.349629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 790/1569] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.351026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.351304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.351572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.364089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 780/1570] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.364089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.364089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.364089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.371409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.372585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.389667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] MFA Rules not processed for user `e01531151f024622b7dae0dcb23aa8f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.390866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.432344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 791/1571] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.433176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.442534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46159c2d-9c8e-4615-ae04-2cd6c3b246d2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.447333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.447935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.448215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.448494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.449138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.449453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.451067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a2d48-4deb-4640-a9bb-a014d2d95583 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.458931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 781/1572] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.460391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.469139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 792/1573] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.469139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.469139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.469139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.471233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc8ad1-0b29-4527-9afd-2c28fca50bce None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.475938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.476569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.476853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.477158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.477834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.480261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.483637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:47.484816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:47.492817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 793/1574] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:47.493531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.503794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126ed45-720b-465c-9589-f1344e154d8a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.508608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.512031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 782/1575] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.512140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.512140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.512319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.512406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.512406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.512406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.547068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 794/1576] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.547790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.573117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105ae6a-8bfc-4148-a010-173e884f3403 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.581925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:47.597426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:47.626729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c1343-659f-4d1d-8c5c-3c686862eb7c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.631334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.631944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.632222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.632490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.633110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.635544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.639496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.640111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.640445 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.640955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.641702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.642041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.643703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.663951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 783/1577] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.664169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 795/1578] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.664284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.664387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.670082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca008d53-1e39-4345-b011-050227e46602 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.674808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.675402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.675683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.675978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.676602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.678487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.709359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 796/1579] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.710728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.719283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857705a-ffdd-42c8-8b53-19fc913e1311 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.767615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.768216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.768496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.768767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.772105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c4d8-7ee5-4be3-a3c6-5cb4d86b40a9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.777030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.777713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.777838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.778023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.778337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.779018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.779227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.780904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.788414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 784/1580] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:47.789275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.799149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addca56e-236e-4910-8e45-91c82a5ec4d9 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.805127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.805731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.806015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.806298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.807514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:47.807822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.809787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.813650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 797/1581] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.813650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.825761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 785/1582] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:47.834774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.845312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed02b281-b13d-4670-9491-bb0a0e02f6c6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.897510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bddf7-3355-4ccc-8f8b-4cb78cb9904b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.904336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.905207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.906131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.906564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.908346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.911345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.925702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:47.937592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.940967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 798/1583] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/users => generated 282 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.941134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.941134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.941285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.941361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.941361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.942984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.953441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afa3b3-8a50-49a4-a600-3812888d797e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.959421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.961311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.961591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.961859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.962882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.964638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:47.968615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 786/1584] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/users => generated 290 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:47.969499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:47.985535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b95f36-1cff-4369-adf2-4a6daf473ff0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:47.992405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:47.993282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:47.993759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:47.994817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:47.995739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:47.997881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.000294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 799/1585] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.000294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.005531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:48.025690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 787/1586] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.026637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.038940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a5e70-44f8-4c95-a43d-57df4f13599b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.082941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c050014a-30ef-457b-ac1b-4719f821c1db admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.088414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.089043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.089309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.089587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.090249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.092732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.116650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.121120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 800/1587] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:47 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.121120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.122284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.123425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.124056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.125013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.127523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.132797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6dc04-d4d8-49ae-a371-3e485efc4ce6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.138771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.139492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.139731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.140008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.140697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.143110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.161106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 788/1588] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/users => generated 303 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.162054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.175941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b75387-2cbf-41ce-ad1c-011ec6457f4b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.181159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.182377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.182763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.183119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.184136 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.185965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.186404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 801/1589] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.187801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.198476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 789/1590] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.199373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.215191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e9abc-fe59-497e-9ebb-4abf2742145e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.266682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6d707c-e5d7-446a-a7d8-d0baedca64a5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.272050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.272866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.273152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.273426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.274081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.276470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.284252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.284883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.285166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.285435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.286199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.290116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 802/1591] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.290116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.290334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.299981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b762d9-3f4a-47b0-b1a5-8a1d7d8967f4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.307960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.308510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.308791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.309178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.309705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.311336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.315836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 790/1592] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.316632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.323108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 803/1593] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.327423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.327820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8f2be-edf1-4986-b8e5-25293c822a9e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.333066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.333771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.334175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.334520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.335305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.336907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.338488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf649a78-ec59-4aea-aac8-6ccd3c87a0d5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.344366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.345251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.345530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.345876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.346728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.348300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 791/1594] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.348447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.349088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.360535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 804/1595] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.361327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f35790-3ed8-4a0e-9742-167a3adcf4d4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.361446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.366356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.367581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.368028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.368450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.369259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.371238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.373752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06beec0-ae55-477a-aeff-4f2fe321da75 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.378295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.378927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.379245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.379472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.380410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.381685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.396222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 805/1596] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.396222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.396531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 792/1597] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:48.396531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.407215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ad610-2589-4f98-935d-a6322a237e0b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.408118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c7bf2-f424-4536-99c5-359210628c85 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.413632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.413870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.414245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.414414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.414532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.414742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.414907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.415012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.415632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.415838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.417345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.417450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.424201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:48.430268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 793/1598] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.431167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.436930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 806/1599] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.436930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.442521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f763fd-3ba4-4b08-b3f1-542c81ca9d5a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.449939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6f074-9aa3-44d5-8ad3-018431b6e87c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.450545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.451585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.451866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.452139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] PATH_INFO: `/v3/projects/507e4c0242624513a740abc56c72d82f/users/bd2f9c9be5ef4b09b78f65e3afa7b8bf/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.455518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.456270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.456551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.456826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] PATH_INFO: `/v3/projects/355d75e2950240dd9a00ac915c202a56/users/68857b40776f431e8ec0bcc0f442a43f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.468015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=507e4c0242624513a740abc56c72d82f, user_id=bd2f9c9be5ef4b09b78f65e3afa7b8bf, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.470946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.474167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355d75e2950240dd9a00ac915c202a56, user_id=68857b40776f431e8ec0bcc0f442a43f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.477056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.485542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:48.487031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 794/1600] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:48 2022] PUT /identity/v3/projects/507e4c0242624513a740abc56c72d82f/users/bd2f9c9be5ef4b09b78f65e3afa7b8bf/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:48.488036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.497569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 807/1601] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:48 2022] PUT /identity/v3/projects/355d75e2950240dd9a00ac915c202a56/users/68857b40776f431e8ec0bcc0f442a43f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:48.498169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.575412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad1946-2034-4694-81d8-01b60f9425a0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.581178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.581893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.582827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.583153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] PATH_INFO: `/v3/projects/9069fb8ae0da44e6af488a709f641ead/users/791e44a94d8a4d64a5cd698c5758c493/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.592601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d88ba-8b82-4dc2-bd4d-d08940f165b4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.602525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.604029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.604163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9069fb8ae0da44e6af488a709f641ead, user_id=791e44a94d8a4d64a5cd698c5758c493, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.604320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.604725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.605445 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.607100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.607846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.620490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 795/1602] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.621272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.626176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 808/1603] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:48 2022] PUT /identity/v3/projects/9069fb8ae0da44e6af488a709f641ead/users/791e44a94d8a4d64a5cd698c5758c493/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:48.627405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.633292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9249007-3bbc-4038-8fe0-d30e8a200020 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.700410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.700648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab044c-4d42-4699-abc3-485008840747 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.701191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.701536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.701965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.702809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.704692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.705835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.706614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.706978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.707324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.708126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.710019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.722566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 796/1604] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.724488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.737329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 809/1605] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.738407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.743427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c4eb-547d-4077-8b14-a6c2119020a8 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.749448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.750610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.750886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.751224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] PATH_INFO: `/v3/projects/450e17cbc2104fcba833fc48ada83cda/users/a59246460c1846ac82200f4516f2507f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.755021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a73e7-0908-426c-b282-1b4c0f20ee92 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.761158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.762029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.762494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.763006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.764228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.766100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70202034-8fd1-4fb5-b059-ae64c2cc8d19 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.769956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=450e17cbc2104fcba833fc48ada83cda, user_id=a59246460c1846ac82200f4516f2507f, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.773032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.779503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 810/1606] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:48.781511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45fe30-2260-4130-939b-bcba4c3f4ddd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.781992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45fe30-2260-4130-939b-bcba4c3f4ddd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.782509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45fe30-2260-4130-939b-bcba4c3f4ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.788254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 797/1607] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:48 2022] PUT /identity/v3/projects/450e17cbc2104fcba833fc48ada83cda/users/a59246460c1846ac82200f4516f2507f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:48.789061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.820818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2f45fe30-2260-4130-939b-bcba4c3f4ddd None None] MFA Rules not processed for user `bd2f9c9be5ef4b09b78f65e3afa7b8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:48.822616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2f45fe30-2260-4130-939b-bcba4c3f4ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:48.855632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:48.878698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da2361-116c-4980-8fff-fd063fd33e42 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:48.885506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:48.886208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.886478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.886830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] PATH_INFO: `/v3/projects/87ff9b0b75894bb0b276080756d85f35/users/a6b6fce66c6f4920a6db2a4d39497e0c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.902045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87ff9b0b75894bb0b276080756d85f35, user_id=a6b6fce66c6f4920a6db2a4d39497e0c, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:48.904763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:48.910418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:48.925527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 811/1609] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:48.927617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 798/1609] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:48 2022] PUT /identity/v3/projects/87ff9b0b75894bb0b276080756d85f35/users/a6b6fce66c6f4920a6db2a4d39497e0c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:48.928338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:48.932166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:48.932470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:48.932738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:48.967562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] MFA Rules not processed for user `791e44a94d8a4d64a5cd698c5758c493`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:48.969168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.024475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba63002-4010-4a55-b38d-3fe1a8356774 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.030289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.031054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.031456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.031822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.032628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.034491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.047469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 799/1610] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:49.052036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.052395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 812/1611] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.056677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.066235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7416c1a-05bb-4b39-aa13-da6cb56b11da None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.071556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.071702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe0759-721b-4daa-84e4-8603ab52035c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.072333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.072647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.073039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] PATH_INFO: `/v3/projects/355d75e2950240dd9a00ac915c202a56/users/68857b40776f431e8ec0bcc0f442a43f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.075883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=355d75e2950240dd9a00ac915c202a56, user_id=68857b40776f431e8ec0bcc0f442a43f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.076791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.077450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.077731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.078008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] PATH_INFO: `/v3/projects/47d9cb4aa3484f95bad54f2233a08439/users/0a30897a10d8467aa294f24954368a84/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.080070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7083dc-e1cb-4eaf-8683-451d9cfd3ade admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.095557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 813/1612] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:49 2022] PUT /identity/v3/projects/355d75e2950240dd9a00ac915c202a56/users/68857b40776f431e8ec0bcc0f442a43f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:49.096902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47d9cb4aa3484f95bad54f2233a08439, user_id=0a30897a10d8467aa294f24954368a84, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.097289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.097588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.097868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.099624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6574f241-3940-45ff-8afd-e9ac521a912a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.127058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 800/1613] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:49 2022] PUT /identity/v3/projects/47d9cb4aa3484f95bad54f2233a08439/users/0a30897a10d8467aa294f24954368a84/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:49.128025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:49.129432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.130831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.131177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.131460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.162885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:49.164209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.208596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 814/1614] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.209361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.221030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d05bc8-a18a-4bfd-a451-0ef5ab5495d5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.226513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.227164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.227444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.227873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.229124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.231458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.258197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 801/1615] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.261400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.267025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 815/1616] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:49.267927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.271559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c863a3a9-ab75-4b93-9f96-886f7492dd02 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.324080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:49.357474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.358764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.359053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.359334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] PATH_INFO: `/v3/projects/c66f397e080f497da06b661ea1a87e03/users/951cabf887f14896b0c4402b686be697/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.363678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f160f8cb-9686-4a4c-bd0b-622ed8396920 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.379550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.380452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.380955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.381715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.388837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:49.391923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66f397e080f497da06b661ea1a87e03, user_id=951cabf887f14896b0c4402b686be697, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.396062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0c391-e294-413b-8086-fd22e37cf26b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.414429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 802/1617] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:49 2022] PUT /identity/v3/projects/c66f397e080f497da06b661ea1a87e03/users/951cabf887f14896b0c4402b686be697/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:49.416619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.416973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.417322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.449496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] MFA Rules not processed for user `68857b40776f431e8ec0bcc0f442a43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:49.450891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.477500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.480331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44fe06-d5b2-4c2c-bb80-69477754cd7d service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.497327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 816/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:49.498934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.499297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.499578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.540022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:49.542125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.576932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 803/1619] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.578609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.671563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 817/1620] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.671563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.693737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13edb07-e2e9-475e-9bcd-2e091b2a6c88 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.699231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.700148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.700615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.700892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.701994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.704034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.712509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b3e0d-6fbe-4813-a1f2-9844b3284235 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.720226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.720857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.721146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.721451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.732687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 818/1621] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/users => generated 308 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:49.733527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.747910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.749384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118d95e-4ea2-409a-8b9d-f0cfaf8cad69 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.756045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5514d-f095-4665-9bc7-f6ff770dc964 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.766528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.767202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.767488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.767767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.771603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 804/1622] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:49.774168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.774459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.774845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.797977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.799447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.811807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 819/1623] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:49.812676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.819485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:49.823884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:49.852268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 805/1624] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:49.852268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.877392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb683af-7a7f-4692-9be7-d65fe159db06 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.882176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.883059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.883350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.883630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.896700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.898461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.911519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0c30c-8354-49a1-943d-d0997f3d2a9f service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.918749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 806/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:49.921533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:49.923329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.924114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.924400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.924674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.937752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d26c0-0fd6-47f9-9f24-774fb1f47d96 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:49.947739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:49.948414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:49.948739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:49.949099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:49.949761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:49.951461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:49.969814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 807/1626] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:49.970913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:49.976412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:50.006020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355cba9-b783-4dcf-9a77-2cb38107594f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:50.015671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:50.016359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.016639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.017145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.036445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.038620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.039235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.043953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.063199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 820/1627] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:50.065383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:50.065597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 808/1628] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:50.081799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e31f8f-b7f5-4056-a80e-788f9b474324 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:50.088476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:50.089164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.089450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.089726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] PATH_INFO: `/v3/projects/240b921f17a54e7a992e91355701ab42/users/b565acfbc26f44ad8d71ab895ea9144f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.107302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] RBAC: Authorizing `identity:create_grant(project_id=240b921f17a54e7a992e91355701ab42, user_id=b565acfbc26f44ad8d71ab895ea9144f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.110196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.130631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 821/1629] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:50 2022] PUT /identity/v3/projects/240b921f17a54e7a992e91355701ab42/users/b565acfbc26f44ad8d71ab895ea9144f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:50.133966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:50.148577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a7b1a-f182-4148-9339-52c5cc98cbeb service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:50.153924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:50.277803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:50.279270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.279563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.279846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.283188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.287442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c002e-5568-4d74-a909-b0b5ba71ce3d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.318333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 809/1630] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:50.319684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:50.339876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd06df4-598c-4bf3-a094-357036fa023f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:50.347039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:50.347844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.348196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.348578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] PATH_INFO: `/v3/projects/240b921f17a54e7a992e91355701ab42/users/b565acfbc26f44ad8d71ab895ea9144f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.354887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] RBAC: Authorizing `identity:create_grant(project_id=240b921f17a54e7a992e91355701ab42, user_id=b565acfbc26f44ad8d71ab895ea9144f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.358530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.395263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 822/1631] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:50 2022] PUT /identity/v3/projects/240b921f17a54e7a992e91355701ab42/users/b565acfbc26f44ad8d71ab895ea9144f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:50.397198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.397599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.397787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.467680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] MFA Rules not processed for user `b565acfbc26f44ad8d71ab895ea9144f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:50.472361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:50.664527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 810/1632] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:50.676264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:50.852519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18f661-f276-4ea7-a781-e1efe00ed46f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:50.864362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:50.864362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:50.867662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:50.868591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:50.910895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:50.917665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff3d32-7478-4bbd-8345-31aeb8fd362f service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:50.966297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 823/1633] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:50 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:51.291470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:51.486819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabc00f-8b52-4176-b965-8af84b712930 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:51.496777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:51.497717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:51.498042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:51.498326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] PATH_INFO: `/v3/users/bd2f9c9be5ef4b09b78f65e3afa7b8bf` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:51.499923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2f9c9be5ef4b09b78f65e3afa7b8bf)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:51.503326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:51.558452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:51.591296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2f9c9be5ef4b09b78f65e3afa7b8bf'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:51.600364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2f9c9be5ef4b09b78f65e3afa7b8bf'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:51.623443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2f9c9be5ef4b09b78f65e3afa7b8bf'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:51.624446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2f9c9be5ef4b09b78f65e3afa7b8bf'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:51.625487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2f9c9be5ef4b09b78f65e3afa7b8bf'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:51.661207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 811/1634] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:12:51 2022] DELETE /identity/v3/users/bd2f9c9be5ef4b09b78f65e3afa7b8bf => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:51.665075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458580b-1dc1-4bb4-b730-5fad57971159 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:51.665544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458580b-1dc1-4bb4-b730-5fad57971159 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:51.665676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458580b-1dc1-4bb4-b730-5fad57971159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:51.688064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c458580b-1dc1-4bb4-b730-5fad57971159 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:51.689445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c458580b-1dc1-4bb4-b730-5fad57971159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:51.806936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 824/1635] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:51.887243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:51.888761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:51.971442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd8f83-9de3-4890-a7c5-d123b9397bdf admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:51.979804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:51.981457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:51.982429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:51.983403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:51.999718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:52.001532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678b6d-a68e-4931-afef-8381ebd06ed7 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:52.019749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 812/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:53.283943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ef86-a320-4e37-b126-622ca535f022 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.284616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ef86-a320-4e37-b126-622ca535f022 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.284616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0ef86-a320-4e37-b126-622ca535f022 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.286477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 825/1637] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 11:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:12:53.296030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae056b3-df1e-42b0-8690-3fde5f87686d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.296566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae056b3-df1e-42b0-8690-3fde5f87686d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.296649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae056b3-df1e-42b0-8690-3fde5f87686d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.385036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3ae056b3-df1e-42b0-8690-3fde5f87686d None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:53.386289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3ae056b3-df1e-42b0-8690-3fde5f87686d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:53.564599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 813/1638] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 11:12:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:53.569964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.571234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.571665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.573232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 826/1639] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 11:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:12:53.593001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.593318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.593739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.633675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:53.639520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:53.654923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 814/1640] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:12:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:53.662149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:53.666720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:53.675224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:53.677330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:53.701669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33649ff-c2a6-44a2-a43a-dce4fc793916 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:53.718636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:53.719560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.720010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.721614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] PATH_INFO: `/v3/projects/507e4c0242624513a740abc56c72d82f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.723293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=507e4c0242624513a740abc56c72d82f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:53.729085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:53.737390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:53.802397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eebadc8-c30a-44f3-928c-a6cbda4a4310 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:53.809046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:53.810206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:53.810814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:53.811294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:53.835848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:53.836629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:53.839834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:53.875593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 815/1641] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:12:53.903445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '507e4c0242624513a740abc56c72d82f', 'request_id': 'req-e0376c9d-d712-446b-924c-b0151e56cc43'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:53.916509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '507e4c0242624513a740abc56c72d82f', 'request_id': 'req-e0376c9d-d712-446b-924c-b0151e56cc43'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:53.930962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '507e4c0242624513a740abc56c72d82f', 'request_id': 'req-e0376c9d-d712-446b-924c-b0151e56cc43'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:53.969756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] The token cache is being invalidate because project 507e4c0242624513a740abc56c72d82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:12:53.970634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:53.973745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e0376c9d-d712-446b-924c-b0151e56cc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:53.987543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 827/1642] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:12:53 2022] DELETE /identity/v3/projects/507e4c0242624513a740abc56c72d82f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:54.000604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:54.031172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2886fc4-9575-4200-a710-c4b0bd440f21 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:54.114956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96c55a-8a79-443c-9a1e-9da39a4213c7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.115497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96c55a-8a79-443c-9a1e-9da39a4213c7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.115994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96c55a-8a79-443c-9a1e-9da39a4213c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.168558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:54.169387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.169525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.169807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] PATH_INFO: `/v3/users/b565acfbc26f44ad8d71ab895ea9144f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.176304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b565acfbc26f44ad8d71ab895ea9144f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:54.179416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:54.186867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1c96c55a-8a79-443c-9a1e-9da39a4213c7 None None] MFA Rules not processed for user `5d1368fb915444b599b3fcf3f5586789`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:54.188147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1c96c55a-8a79-443c-9a1e-9da39a4213c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:54.293385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b565acfbc26f44ad8d71ab895ea9144f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:54.310407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b565acfbc26f44ad8d71ab895ea9144f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:54.334102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b565acfbc26f44ad8d71ab895ea9144f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:54.334752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 828/1643] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:12:54.335891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b565acfbc26f44ad8d71ab895ea9144f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:54.337147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4d64ad05-061d-4316-9118-70765451315d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b565acfbc26f44ad8d71ab895ea9144f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:54.338404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8544b6-9489-40a1-807a-7422c060a3bb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.339237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8544b6-9489-40a1-807a-7422c060a3bb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.340147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8544b6-9489-40a1-807a-7422c060a3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.432399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 816/1644] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:12:53 2022] DELETE /identity/v3/users/b565acfbc26f44ad8d71ab895ea9144f => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:54.434992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-db8544b6-9489-40a1-807a-7422c060a3bb None None] MFA Rules not processed for user `5d1368fb915444b599b3fcf3f5586789`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:54.434992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-db8544b6-9489-40a1-807a-7422c060a3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:54.649078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 829/1645] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:54.703234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.703924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.704222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.706235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 817/1646] 10.222.0.49 () {54 vars in 962 bytes} [Wed Aug 10 11:12:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:12:54.716370 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.716619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.716892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.753913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:54.755698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:54.834685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:54.893513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 830/1647] 10.222.0.49 () {58 vars in 1059 bytes} [Wed Aug 10 11:12:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:54.901741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:54.903806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:54.907775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:54.923847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa215a8-a127-4167-ba11-1f7c8a5d022a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:54.930163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:54.930814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:54.931093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:54.932257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:54.934078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:54.951424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:54.954735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b9553-ed53-4678-8d1b-a21e53d05f85 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:54.965781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 831/1648] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:12:54.995633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:55.035552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec407932-1bf7-4bfd-8751-5f20a9347508 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:55.062510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:55.063703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.065346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.065754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] PATH_INFO: `/v3/projects/240b921f17a54e7a992e91355701ab42` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.070668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=240b921f17a54e7a992e91355701ab42)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:55.072013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: WARNING py.warnings [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:55.072013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: warnings.warn(msg) Aug 10 11:12:55.072013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: Aug 10 11:12:55.076049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:55.219901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '240b921f17a54e7a992e91355701ab42', 'request_id': 'req-12121767-742c-485c-aead-7924eb748d26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:55.229837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '240b921f17a54e7a992e91355701ab42', 'request_id': 'req-12121767-742c-485c-aead-7924eb748d26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:55.249435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '240b921f17a54e7a992e91355701ab42', 'request_id': 'req-12121767-742c-485c-aead-7924eb748d26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:55.282792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] The token cache is being invalidate because project 240b921f17a54e7a992e91355701ab42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:12:55.283088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:55.283771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-12121767-742c-485c-aead-7924eb748d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:55.293035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 818/1649] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:12:54 2022] DELETE /identity/v3/projects/240b921f17a54e7a992e91355701ab42 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:55.296568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aa88-0ad6-4283-890d-d03823d63f4a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.297066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aa88-0ad6-4283-890d-d03823d63f4a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.297491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680aa88-0ad6-4283-890d-d03823d63f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.351554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0680aa88-0ad6-4283-890d-d03823d63f4a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:55.354463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0680aa88-0ad6-4283-890d-d03823d63f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.426065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.426566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.426672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.481528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] MFA Rules not processed for user `a6b6fce66c6f4920a6db2a4d39497e0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:55.483730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.494774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 832/1650] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:55.496749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3752c-e6d6-49cb-85c3-cd9e36a629a2 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.497242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3752c-e6d6-49cb-85c3-cd9e36a629a2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.497780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3752c-e6d6-49cb-85c3-cd9e36a629a2 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.501683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 833/1651] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 11:12:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:12:55.503142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf636-a804-43e9-92e8-990ab797ecd1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.503630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf636-a804-43e9-92e8-990ab797ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.504175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cf636-a804-43e9-92e8-990ab797ecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.533653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-116cf636-a804-43e9-92e8-990ab797ecd1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:55.535288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-116cf636-a804-43e9-92e8-990ab797ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.545622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 834/1652] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:55.548804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.549324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.550418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.597222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:55.600642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.617254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 819/1653] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:12:55.619392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:55.660660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fad09-b8fe-4471-9975-f7ac92be3873 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:55.683065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:55.684058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.684471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.684862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.685662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:12:55.688054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:55.692085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:55.731658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 820/1654] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:12:55.737246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:55.746020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.765872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86689989-0110-4b0a-95fc-ede41f93a4d1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:55.771204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:55.776390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:55.777295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:55.778536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:55.779830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:55.783505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:55.839468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 835/1655] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:55.840604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:55.859667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 821/1656] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:55.867384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76b86a-2c79-4b7c-84b2-363ddbaac6c8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:55.876157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:55.880830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:55.955912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:56.028150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:56.047799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.049350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.050893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.051321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.052514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.059943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.079640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe666da-795f-4eb7-a2cb-c2e6c0115744 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.089933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.090687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.091434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.093334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.138719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 836/1657] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:55 2022] POST /identity/v3/projects => generated 365 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:56.140087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.279810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49e5ec-2152-43e7-a2f1-974282ca9a69 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.289755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.291152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.291278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.291533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.292188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.294121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.311353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.312826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.336529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 837/1658] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:56 2022] POST /identity/v3/users => generated 309 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:56.341703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.342740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 822/1659] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:56.344144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.369004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06fa69-1013-4626-ad00-282fd592bd88 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.374270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3530c-3902-43c5-b7ac-2967f5ff72a8 service glance] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.380427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.382699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.383302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.383987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.386165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.387606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.389513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.392377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.392890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.393396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.396258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.421052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.462992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 823/1660] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:56.466477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.510036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa67b6-927d-429c-82a3-2de9075368c2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.520941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 838/1661] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:56 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:56.521948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.523813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.526468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.526978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.528332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] PATH_INFO: `/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4/users/9167304fee3f4fc5a37f4825cb60f3c4/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.547222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c6545e-8e50-4559-bc28-158be39d0239 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.565931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.566563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.566891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.567424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.567814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.573064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.595562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd32b3d663f4c80a0a04b75fb0a4bb4, user_id=9167304fee3f4fc5a37f4825cb60f3c4, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.600890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.611139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 839/1662] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:56.612592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.658714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 824/1663] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:56 2022] PUT /identity/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4/users/9167304fee3f4fc5a37f4825cb60f3c4/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:56.661402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.674731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f1be40-0e83-4aad-9d8e-5119f5429992 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.814400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.815273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.815554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.815825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.816482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.818093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.855848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 840/1664] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:56.856777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.889488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64a0db8-bce9-4546-8c0e-3b2fc2ace7c6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.894285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7f2a-b0a6-43fc-9c95-da55814d2336 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:56.919929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.920617 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.925732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.926000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] PATH_INFO: `/v3/projects/050edb381d0143f2b770b661550ec237/users/ed03dddc55994b16a80afdcdec5e3c14/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.931376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:56.932027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:56.932312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:56.932570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:56.938149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.944498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:56.975418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 825/1665] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:12:56.975644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:56.977873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=050edb381d0143f2b770b661550ec237, user_id=ed03dddc55994b16a80afdcdec5e3c14, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:56.993421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d71a08-4db6-418b-93b3-a5926b7f8f7e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.010410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04062780-4a83-45e0-a189-e5c929a67487 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.034610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 841/1666] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:56 2022] PUT /identity/v3/projects/050edb381d0143f2b770b661550ec237/users/ed03dddc55994b16a80afdcdec5e3c14/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:57.036161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.036449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.036723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.037832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:57.105752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] MFA Rules not processed for user `ed03dddc55994b16a80afdcdec5e3c14`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:57.107026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:57.116687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:57.157012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.157700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.157981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.158257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] PATH_INFO: `/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4/users/9167304fee3f4fc5a37f4825cb60f3c4/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.160391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd32b3d663f4c80a0a04b75fb0a4bb4, user_id=9167304fee3f4fc5a37f4825cb60f3c4, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.163141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.195460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 826/1667] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:56 2022] PUT /identity/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4/users/9167304fee3f4fc5a37f4825cb60f3c4/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:57.256007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 842/1668] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:57.258976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.345911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3331cdd-c275-486a-9f7f-74a267123fe1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.356821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.357043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.357932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.357932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.358444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.361074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c03fd-31d4-44ed-b2b2-1f0b99876540 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.410781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 827/1669] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:12:57 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:57.418153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.426758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e913b-3460-4936-a033-78e050803bac None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.427256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e913b-3460-4936-a033-78e050803bac None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.427632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e913b-3460-4936-a033-78e050803bac None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.429813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 828/1670] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 11:12:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:12:57.438091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.438383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.438691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.440756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035696a-92ef-4b87-88fb-2f35a7996543 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.506122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:57.508018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:57.533602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.534608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.534903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.535168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.535824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.537757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.597804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 843/1671] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:12:57 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:12:57.598754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.613404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859a02f-b69b-4f08-add3-d19dec6b6762 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.618701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 829/1672] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:12:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 10 11:12:57.629940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.630871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.631236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.631581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.632322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.633834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.634266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.636128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:57.651556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 844/1673] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:57.652707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.654916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410f018-6c36-4b75-8331-8e3d97ea91eb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.680826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.682124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.682354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.682654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.686346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c2c3d-7312-4852-8ea9-57a2a3670d3f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.700349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.701149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.701497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.702327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] PATH_INFO: `/v3/projects/481a9221d88240e28d352a4760e2c0ce/users/e7220a2b44cd4eca85b12c46d0954d60/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.703144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.709004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.724836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 830/1674] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:57.735451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] RBAC: Authorizing `identity:create_grant(project_id=481a9221d88240e28d352a4760e2c0ce, user_id=e7220a2b44cd4eca85b12c46d0954d60, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.739355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.763339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 845/1675] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:57 2022] PUT /identity/v3/projects/481a9221d88240e28d352a4760e2c0ce/users/e7220a2b44cd4eca85b12c46d0954d60/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:57.764788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.784071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb1236a-198e-4919-be87-4e067e6c736c service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:57.800214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:57.938544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:57.939187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:57.939496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:57.939774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:57.940492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:57.942599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932d502-e722-48a0-a410-ad5293e59504 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:57.960306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 831/1676] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:12:57.964858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:57.988932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5fe17-c8d8-4d58-b87c-117ddc52460a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:58.011683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:58.012660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:58.012970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:58.015500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] PATH_INFO: `/v3/projects/481a9221d88240e28d352a4760e2c0ce/users/e7220a2b44cd4eca85b12c46d0954d60/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:58.025163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=481a9221d88240e28d352a4760e2c0ce, user_id=e7220a2b44cd4eca85b12c46d0954d60, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:58.032310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:58.053892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 846/1677] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:12:57 2022] PUT /identity/v3/projects/481a9221d88240e28d352a4760e2c0ce/users/e7220a2b44cd4eca85b12c46d0954d60/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:12:58.067528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:58.067922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:58.068396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:58.119409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] MFA Rules not processed for user `e7220a2b44cd4eca85b12c46d0954d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:58.123791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:58.332010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 832/1678] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:58.343719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:58.345924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:58.519644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:58.530916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6b907-d9ae-4790-9192-81a571c4c309 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:58.535176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:58.535924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:58.536971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:58.536971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:58.551110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:58.552927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:58.572285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 847/1679] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:59.449841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:59.451501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:59.549129 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:12:59.660258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0629854b-bd19-4c26-ad47-ae58ef533010 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:59.678365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:59.679597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:59.680006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:59.680386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:59.686021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:59.686558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7927eb5-a00f-450b-be5a-7e7c8aeaf916 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:12:59.693531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:12:59.694463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:59.694916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:59.695322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] PATH_INFO: `/v3/users/ed03dddc55994b16a80afdcdec5e3c14` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:59.696616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed03dddc55994b16a80afdcdec5e3c14)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:59.698074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: WARNING py.warnings [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:12:59.698074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: warnings.warn(msg) Aug 10 11:12:59.698074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: Aug 10 11:12:59.699753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:59.714714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:12:59.717840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c40e1-d745-4c26-a971-e0b4eb190152 service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:12:59.761447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 833/1680] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:12:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:12:59.769388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945f3c-77ac-47a3-9005-1a8fd3656ea5 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:59.769887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945f3c-77ac-47a3-9005-1a8fd3656ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:59.770322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945f3c-77ac-47a3-9005-1a8fd3656ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:12:59.835651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-21945f3c-77ac-47a3-9005-1a8fd3656ea5 None None] MFA Rules not processed for user `5d1368fb915444b599b3fcf3f5586789`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:12:59.837323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-21945f3c-77ac-47a3-9005-1a8fd3656ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:12:59.879883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed03dddc55994b16a80afdcdec5e3c14'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:59.890344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:12:59.913535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed03dddc55994b16a80afdcdec5e3c14'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:59.929797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed03dddc55994b16a80afdcdec5e3c14'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:59.930565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed03dddc55994b16a80afdcdec5e3c14'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:59.931306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed03dddc55994b16a80afdcdec5e3c14'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:12:59.984995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 834/1681] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:12:59.988657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 848/1682] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:12:59 2022] DELETE /identity/v3/users/ed03dddc55994b16a80afdcdec5e3c14 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:12:59.992570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123499-5846-4f79-a089-bfd0c390c1e4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:12:59.993040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123499-5846-4f79-a089-bfd0c390c1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:12:59.993853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61123499-5846-4f79-a089-bfd0c390c1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:00.035015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-61123499-5846-4f79-a089-bfd0c390c1e4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:00.039391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-61123499-5846-4f79-a089-bfd0c390c1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:00.246692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 835/1683] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:00.267634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:00.270321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:00.395569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e46998-8d92-4bad-9dab-c281ad224d50 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:00.400405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:00.400964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:00.401255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:00.401692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:00.416895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:00.418351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:00.432010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 849/1684] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:01.576175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d16340-3826-453a-8c8b-12c080ef596a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:01.576691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d16340-3826-453a-8c8b-12c080ef596a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:01.576795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d16340-3826-453a-8c8b-12c080ef596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:01.685270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:01.687564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-57d16340-3826-453a-8c8b-12c080ef596a None None] MFA Rules not processed for user `9167304fee3f4fc5a37f4825cb60f3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:01.687847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:01.689317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-57d16340-3826-453a-8c8b-12c080ef596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:01.743485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:01.906405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 836/1685] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:01.975710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623dffb-052e-441b-9c62-b1fee5409bd8 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:01.988805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:01.989629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:01.990069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:01.990493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:01.997083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:02.249223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:02.251105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:02.289751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 850/1686] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:01 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:03.150369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:03.151022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:03.151022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:03.187748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:03.189894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:03.207754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a6297-2986-4c23-9292-edd710f9ebfd service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:03.230058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:03.230908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:03.232362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:03.232922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] PATH_INFO: `/v3/projects/050edb381d0143f2b770b661550ec237` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:03.235572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=050edb381d0143f2b770b661550ec237)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:03.242837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:03.289009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:03.291446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:03.337130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:03.364262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050edb381d0143f2b770b661550ec237', 'request_id': 'req-d3afbe01-c878-45ae-be85-c8b01e060ac9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.385921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050edb381d0143f2b770b661550ec237', 'request_id': 'req-d3afbe01-c878-45ae-be85-c8b01e060ac9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.404679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050edb381d0143f2b770b661550ec237', 'request_id': 'req-d3afbe01-c878-45ae-be85-c8b01e060ac9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.424369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 837/1687] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:13:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:03.438312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] The token cache is being invalidate because project 050edb381d0143f2b770b661550ec237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:03.439845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.440936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.451148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 851/1688] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:03 2022] DELETE /identity/v3/projects/050edb381d0143f2b770b661550ec237 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:03.461173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:03.499516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a43ed2-4f48-44ab-8e42-4061e5284525 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:03.583809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:03.584737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:03.584867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:03.585122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] PATH_INFO: `/v3/users/e7220a2b44cd4eca85b12c46d0954d60` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:03.586602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7220a2b44cd4eca85b12c46d0954d60)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:03.589302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:03.684648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7220a2b44cd4eca85b12c46d0954d60'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.705781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7220a2b44cd4eca85b12c46d0954d60'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.720914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7220a2b44cd4eca85b12c46d0954d60'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.722308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7220a2b44cd4eca85b12c46d0954d60'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.723164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b3cb8fd7-09ab-48f4-812e-b79d275b328c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7220a2b44cd4eca85b12c46d0954d60'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:03.788545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 838/1689] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:03 2022] DELETE /identity/v3/users/e7220a2b44cd4eca85b12c46d0954d60 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:03.829837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:03.831885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:03.940640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:04.010065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3afbe01-c878-45ae-be85-c8b01e060ac9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:04.177144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:04.178295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:04.178295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:04.178500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:04.181513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:04.326885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:04.348603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaca73b-e96e-4a9e-8993-b5093aaaaf6e service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:04.376971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 852/1690] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:13:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:13:05.391476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:05.393581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:05.393996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:05.395819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 839/1691] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:13:05 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:13:05.446252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:05.448316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:05.448814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:05.501862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:05.504296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:05.701774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 853/1692] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:13:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:05.831507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:05.833756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:05.883162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e8f42-18ff-4b72-a5bb-2b8a79e0d64f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:05.897073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:05.898311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:05.898835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:05.899254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:05.912621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:05.914712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8a9f8-dcc5-42d5-9773-2beb43760265 service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:05.930256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 840/1693] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:06.791493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:06.794021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:06.808283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:06.965588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3aca-5f49-432a-a6ad-fe5118305fae None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:06.970681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:06.971564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:06.972093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:06.972798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] PATH_INFO: `/v3/projects/481a9221d88240e28d352a4760e2c0ce` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:06.974156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=481a9221d88240e28d352a4760e2c0ce)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:06.976332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:07.062070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '481a9221d88240e28d352a4760e2c0ce', 'request_id': 'req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:07.070480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '481a9221d88240e28d352a4760e2c0ce', 'request_id': 'req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:07.091512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '481a9221d88240e28d352a4760e2c0ce', 'request_id': 'req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:07.126908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] The token cache is being invalidate because project 481a9221d88240e28d352a4760e2c0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:07.127705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:07.128538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-805fa2dd-b9cb-43c0-ae39-e61bc8fe2967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:07.134956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 854/1694] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:06 2022] DELETE /identity/v3/projects/481a9221d88240e28d352a4760e2c0ce => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:07.135888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.136367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.136451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.192854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:07.194360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:07.269242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:07.340506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 841/1695] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:07.350297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.350601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.350927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.395943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:07.397205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:07.417727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:07.418259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 855/1696] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:07.441482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a66964-9078-467b-947b-2bcbb4bdcf88 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:07.476178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:07.477008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.477549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.477969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.480681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:07.481324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:07.488252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:07.519061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 842/1697] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:07.522440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:07.532257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c7a6d-a57a-444e-8f7d-1dc023e2f3d6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:07.568998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:07.569635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.569925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.570204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.570978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:07.579629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080077b-70f4-47f7-9683-c92762136e00 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:07.678198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 856/1698] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:07 2022] POST /identity/v3/projects => generated 355 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:07.687990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:07.717715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13fa29-98a2-4cf5-9dd6-fc4032f43182 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:07.757650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.758229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.759020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.816398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] MFA Rules not processed for user `0a30897a10d8467aa294f24954368a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:07.822257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:07.884465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:07.885101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:07.885381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:07.885653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:07.886281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:07.903553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:07.970438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 843/1699] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:07 2022] POST /identity/v3/users => generated 278 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:07.974523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:07.996309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef0d30e-d4aa-42b6-9ba8-0d5da733c0db admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:08.033901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:08.034493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:08.035658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.037648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.037942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.038737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:08.043192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:08.058020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 857/1700] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:08.076504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:08.079783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:08.082076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 844/1701] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:08.085822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:08.110089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf882df-40ec-4cee-8586-0fbdd4036f15 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:08.116947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:08.119045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.119045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.119045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.119045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:08.134959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:08.161000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 845/1702] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:08.166098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:08.213991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b34ca0-e287-4d04-be27-bd108c9c7f07 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:08.229573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:08.230325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.230634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.230949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] PATH_INFO: `/v3/projects/801934b94e6843169862f95a88203d0c/users/3acb858957604c7b8cf1789d330e59d4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.288179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=801934b94e6843169862f95a88203d0c, user_id=3acb858957604c7b8cf1789d330e59d4, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:08.288826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65a28e-baf2-48c7-8626-cec6ee685824 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:08.296756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:08.297564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.297939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.298381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.301679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b2380-6683-4708-b36d-5ea714704d62 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:08.311769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:08.313853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:08.345128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 846/1703] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:08 2022] PUT /identity/v3/projects/801934b94e6843169862f95a88203d0c/users/3acb858957604c7b8cf1789d330e59d4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:08.354408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.354741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.355021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.367466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 858/1704] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:08.426310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] MFA Rules not processed for user `3acb858957604c7b8cf1789d330e59d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:08.429478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:08.605227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:08.626613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 847/1705] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:08.630990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:08.653890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abeb462-bc51-4b06-8168-f9058bc2520e service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:08.890421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:08.891074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:08.900969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:08.902490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:08.906050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:08.912972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:09.008286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:09.022601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 859/1706] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:08 2022] POST /identity/v3/projects => generated 357 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:09.064137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f259ed75-1a3b-4060-a33e-506ae0ff1a6c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:09.275778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:09.276690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.277148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.278077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.279047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:09.283175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:09.351173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 848/1707] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:09 2022] POST /identity/v3/users => generated 285 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:09.355535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:09.397743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d08214-e4a0-4873-96d1-f309757e33b8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:09.415575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:09.415575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.415575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.415575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.416409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:09.416409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:09.450659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:09.457086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 860/1708] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:09.482007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31910d94-e6a8-49e1-a79b-1fbe20f87a40 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:09.502484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:09.503225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.503516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.508026 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] PATH_INFO: `/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398/users/92b66460ba6549bfbf71852953bcc9a7/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.593881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b066d71fee1f4ab1b4382f9e8d2aa398, user_id=92b66460ba6549bfbf71852953bcc9a7, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:09.610779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:09.657238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 849/1709] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:09 2022] PUT /identity/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398/users/92b66460ba6549bfbf71852953bcc9a7/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:09.661375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:09.682676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d1c0d-01ca-4b35-8a05-b704bcff85e0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:09.842940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:09.860914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:09.865016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.865549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.866050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.868354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:09.872309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d82088-1191-40a2-8063-0268a41badf1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:09.898358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 861/1710] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:09.899655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:09.937692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda77623-57e1-40f2-b512-490ba54bbbe2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:09.965434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:09.966148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.966443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.969791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] PATH_INFO: `/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398/users/92b66460ba6549bfbf71852953bcc9a7/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.984141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b066d71fee1f4ab1b4382f9e8d2aa398, user_id=92b66460ba6549bfbf71852953bcc9a7, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:09.990350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703df3-c3e9-45ee-915b-9dfbed91e2d5 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:09.990993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703df3-c3e9-45ee-915b-9dfbed91e2d5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:09.991480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703df3-c3e9-45ee-915b-9dfbed91e2d5 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:09.993332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 862/1711] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:13:10.000749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93fafa3-6ffc-4bdb-8164-c92ce65869dd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:10.049667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 850/1712] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:09 2022] PUT /identity/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398/users/92b66460ba6549bfbf71852953bcc9a7/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:10.062816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.066953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.067389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.170929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d157-9a29-4201-96bf-d8c1958135a8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.171259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d157-9a29-4201-96bf-d8c1958135a8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.171562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de82d157-9a29-4201-96bf-d8c1958135a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.176675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] MFA Rules not processed for user `92b66460ba6549bfbf71852953bcc9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:10.177880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.217673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-de82d157-9a29-4201-96bf-d8c1958135a8 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:10.219121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-de82d157-9a29-4201-96bf-d8c1958135a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.373726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 863/1713] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:10.376075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:10.380041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.383698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 851/1714] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:10.448063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437964-8d05-4587-ab27-8c33ca37ea7b None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.448430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437964-8d05-4587-ab27-8c33ca37ea7b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.449004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a437964-8d05-4587-ab27-8c33ca37ea7b None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.450363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 852/1715] 10.222.0.49 () {54 vars in 961 bytes} [Wed Aug 10 11:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:13:10.455971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.456394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.456738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.501559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] MFA Rules not processed for user `05f5ea0d51694d8796b40d547d9a0474`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:10.504077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.528708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 853/1716] 10.222.0.49 () {58 vars in 1058 bytes} [Wed Aug 10 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:10.536356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:10.545980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35795fc8-6d82-48c4-8f3f-0155459b30a2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:10.546669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41133ad3-6bea-4ad7-8f80-336b9c01e537 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:10.564159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:10.564814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.565100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.565382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.567923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:10.568557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.568978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.569257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.586947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:10.589208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c7502-d140-4f76-91f8-5b0dfb016c48 service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:10.596326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:10.600367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:10.617979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 854/1717] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:10.650788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 864/1718] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:10 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:10.733050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffe48e-b97d-466b-adb8-a8c553c70e85 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:10.733904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffe48e-b97d-466b-adb8-a8c553c70e85 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:10.733904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffe48e-b97d-466b-adb8-a8c553c70e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:10.799292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-67ffe48e-b97d-466b-adb8-a8c553c70e85 None None] MFA Rules not processed for user `a59246460c1846ac82200f4516f2507f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:10.800593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-67ffe48e-b97d-466b-adb8-a8c553c70e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.842436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:10.922961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:10.926060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:10.997305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 855/1719] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:11.000488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:11.000895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:11.001198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:11.036006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] MFA Rules not processed for user `a59246460c1846ac82200f4516f2507f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:11.038765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:11.061847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 856/1720] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:11.061847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:11.095401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2b49b-e57c-4ecc-82ab-36b135770983 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:11.100178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:11.100902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:11.101322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:11.102692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:11.221890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d05c7e-5c22-488f-b505-1b1079d0c6c2 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:11.236000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:11.236784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:11.237170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:11.237552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] PATH_INFO: `/v3/users/a6b6fce66c6f4920a6db2a4d39497e0c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:11.246184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:11.250917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6b6fce66c6f4920a6db2a4d39497e0c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:11.255373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:11.269600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:11.288163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 857/1721] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:11.289409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:11.293389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:11.351607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:11.422126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6b6fce66c6f4920a6db2a4d39497e0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:11.456244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6b6fce66c6f4920a6db2a4d39497e0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:11.478521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6b6fce66c6f4920a6db2a4d39497e0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:11.480812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6b6fce66c6f4920a6db2a4d39497e0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:11.486113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6c05c278-9c74-42e3-b738-67095d8f5d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6b6fce66c6f4920a6db2a4d39497e0c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:11.523764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f437d5-b5fe-438d-8a2a-5e86c5bae7f4 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:11.537224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 865/1722] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:10 2022] DELETE /identity/v3/users/a6b6fce66c6f4920a6db2a4d39497e0c => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:11.548073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:11.549282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:11.551510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:11.554983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:11.558981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:11.590069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:11.592184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:11.616095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 858/1723] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:12.946251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:12.947326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:12.947958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:13.028890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] MFA Rules not processed for user `951cabf887f14896b0c4402b686be697`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:13.031668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:13.063025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:13.069599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:13.102477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68697e83-754c-4c0b-b05c-c7ccfc9ebd86 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:13.186286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:13.204021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 866/1724] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:13.220216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:13.221729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:13.285498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:13.288054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:13.288893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:13.289276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] PATH_INFO: `/v3/projects/87ff9b0b75894bb0b276080756d85f35` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:13.290701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87ff9b0b75894bb0b276080756d85f35)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:13.294289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:13.403268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b84e53-8ac3-4d09-92d6-29bafdaec57b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:13.409308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87ff9b0b75894bb0b276080756d85f35', 'request_id': 'req-4088141a-772b-4970-b2f1-818b089df7c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.434044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87ff9b0b75894bb0b276080756d85f35', 'request_id': 'req-4088141a-772b-4970-b2f1-818b089df7c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.452710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87ff9b0b75894bb0b276080756d85f35', 'request_id': 'req-4088141a-772b-4970-b2f1-818b089df7c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.484700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:13.484871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] The token cache is being invalidate because project 87ff9b0b75894bb0b276080756d85f35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:13.485111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.486722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.492969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 859/1725] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:13 2022] DELETE /identity/v3/projects/87ff9b0b75894bb0b276080756d85f35 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:13.496795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:13.501752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:13.575544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:13.576958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:13.577447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:13.578202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:13.580585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:13.645413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088141a-772b-4970-b2f1-818b089df7c4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:13.658812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:13.659475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:13.659760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:13.660042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] PATH_INFO: `/v3/users/9167304fee3f4fc5a37f4825cb60f3c4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:13.661239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9167304fee3f4fc5a37f4825cb60f3c4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:13.663240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:13.681081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:13.683859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:13.698920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 867/1726] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:13.785861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9167304fee3f4fc5a37f4825cb60f3c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.807257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9167304fee3f4fc5a37f4825cb60f3c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.814148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:13.821001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9167304fee3f4fc5a37f4825cb60f3c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.821881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9167304fee3f4fc5a37f4825cb60f3c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.822795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fb9e5eb-1617-4e2e-840a-23752370c38e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9167304fee3f4fc5a37f4825cb60f3c4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:13.879211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 860/1727] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:13 2022] DELETE /identity/v3/users/9167304fee3f4fc5a37f4825cb60f3c4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:14.267722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:14.285886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d6a2d-b882-4f20-b07c-5ab3fa00c181 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:14.402651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:14.403838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:14.404136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:14.405362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] PATH_INFO: `/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:14.407532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd32b3d663f4c80a0a04b75fb0a4bb4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:14.410052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:14.567101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd32b3d663f4c80a0a04b75fb0a4bb4', 'request_id': 'req-3046adfd-8211-4f0e-a45b-607dfbc4969b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:14.592968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd32b3d663f4c80a0a04b75fb0a4bb4', 'request_id': 'req-3046adfd-8211-4f0e-a45b-607dfbc4969b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:14.619895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd32b3d663f4c80a0a04b75fb0a4bb4', 'request_id': 'req-3046adfd-8211-4f0e-a45b-607dfbc4969b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:14.622135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:14.656042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] The token cache is being invalidate because project fdd32b3d663f4c80a0a04b75fb0a4bb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:14.657170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:14.657170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3046adfd-8211-4f0e-a45b-607dfbc4969b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:14.666328 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 868/1728] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:14 2022] DELETE /identity/v3/projects/fdd32b3d663f4c80a0a04b75fb0a4bb4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:14.668072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:14.668558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:14.668747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:14.711595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:14.715637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:14.810412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 861/1729] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:14.818524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:14.818913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:14.819255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:14.868634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:14.870112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:14.890077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 869/1730] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:14.893518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:14.908677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769ade1-c6ad-46cb-b950-4dd6b6e7334e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:14.919550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:14.920241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:14.920532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:14.920812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:14.921620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:14.921949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:14.927583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:14.956641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 862/1731] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:14.963193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:15.002627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0b96f-5f77-4aca-86a1-a33642922633 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:15.036231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:15.036607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.036889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.037310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.038288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:15.053052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:15.175385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 870/1732] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:14 2022] POST /identity/v3/projects => generated 369 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:15.179852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:15.202231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ed3c0-d87e-4b4d-8f5c-a0f44ce3dc42 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:15.420599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:15.421241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.421525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.421801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.438682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:15.441016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:15.498924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 863/1733] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:15 2022] POST /identity/v3/users => generated 285 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:15.501762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:15.542931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042adac6-10f9-4e19-86c9-fc4eb0f4b6e8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:15.561711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:15.564086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.564372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.564646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.565313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:15.571638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:15.615847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 871/1734] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:15.623321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:15.646397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c35fcd-d4f0-47bd-937e-5bcc1f9cbb10 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:15.674428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:15.675577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.675859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.676118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.676953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:15.678381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06023bc4-f6a4-4c00-9487-ca9f0df59cef admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:15.729820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 864/1735] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:15.732265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:15.758478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4471-e6e7-440b-af51-f6f03be5e663 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:15.781761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:15.788103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.788455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.788738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] PATH_INFO: `/v3/projects/5544c57af7a24b389090f6537e3ae941/users/2b994de001584ce2b98e3e10228a9718/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.834483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5544c57af7a24b389090f6537e3ae941, user_id=2b994de001584ce2b98e3e10228a9718, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:15.841121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e4d47a-5c51-49f4-bc34-52e841974929 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:15.883121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 872/1736] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:15 2022] PUT /identity/v3/projects/5544c57af7a24b389090f6537e3ae941/users/2b994de001584ce2b98e3e10228a9718/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:15.895837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.896137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.896656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.948461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] MFA Rules not processed for user `2b994de001584ce2b98e3e10228a9718`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:15.950004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:15.959682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:15.959968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:15.960339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:15.984197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] MFA Rules not processed for user `0a30897a10d8467aa294f24954368a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:15.986066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:16.027788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:16.110811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 865/1737] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:16.111937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:16.125583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 873/1738] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:16.255289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd910f-20b6-4fe8-a659-1d73971fbada None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:16.266177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:16.266960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:16.272585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:16.273132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:16.273945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:16.276957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:16.338908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 866/1739] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:16 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:16.343261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:16.361605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be7240-8e43-42f2-a54a-d8cde34f797b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:16.461121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:16.461915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:16.462365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:16.462766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:16.463526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:16.465764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:16.504450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:16.507060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 874/1740] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:16 2022] POST /identity/v3/users => generated 291 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:16.532618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fd71c-f281-46a1-9212-ec37ae1b8c74 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:16.579339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:16.579966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:16.580239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:16.580714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:16.581389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:16.588141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:16.662232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 867/1741] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:16.663724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:16.694209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386a95-237c-4abe-959a-2ae71f78a2e8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:16.708920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:16.709844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:16.711196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:16.711633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] PATH_INFO: `/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1/users/ecfb735713574d3882d6417d71f21cfc/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:16.785906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3024da08d4a46aeb55cb2ca5b2277a1, user_id=ecfb735713574d3882d6417d71f21cfc, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:16.793740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:16.833467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:16.866085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 875/1742] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:16 2022] PUT /identity/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1/users/ecfb735713574d3882d6417d71f21cfc/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:16.870042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:16.896653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d4d8fc-afed-4d3f-a258-0e3f98ba8093 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:17.165289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:17.165887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:17.166171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:17.166448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:17.167198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:17.168860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e01b7-bf05-46ed-9d29-7337514bea25 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:17.200129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 868/1743] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:17.205227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:17.227294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ac0f6-c5fd-4098-8cd3-d5f6e6745ba5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:17.248576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:17.249414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:17.250511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:17.252064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] PATH_INFO: `/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1/users/ecfb735713574d3882d6417d71f21cfc/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:17.272883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3024da08d4a46aeb55cb2ca5b2277a1, user_id=ecfb735713574d3882d6417d71f21cfc, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:17.288423 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f9752-9b23-4f8c-934d-9165316179b6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:17.339849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 876/1744] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:17 2022] PUT /identity/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1/users/ecfb735713574d3882d6417d71f21cfc/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:17.350383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:17.350711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:17.350997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:17.489974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] MFA Rules not processed for user `ecfb735713574d3882d6417d71f21cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:17.500994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:17.688397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689eb1-5227-4a74-9c5c-9b94df1a4bbe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:17.688861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689eb1-5227-4a74-9c5c-9b94df1a4bbe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:17.689031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689eb1-5227-4a74-9c5c-9b94df1a4bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:17.737962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:17.754928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-79689eb1-5227-4a74-9c5c-9b94df1a4bbe None None] MFA Rules not processed for user `951cabf887f14896b0c4402b686be697`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:17.756193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-79689eb1-5227-4a74-9c5c-9b94df1a4bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:17.862161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 869/1745] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:17.876561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:17.878124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:17.935752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 877/1746] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:18.036891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8cc10d-f244-4a24-a060-5caceeeab081 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:18.042222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:18.044014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:18.044374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:18.045239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:18.059134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:18.065241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:18.065558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:18.065940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:18.092792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:18.172528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 870/1747] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:18.193174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:18.194768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:18.313308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:18.314868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:18.427393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 878/1748] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Aug 10 11:13:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:18.448000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:18.449890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:18.477319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0de872-29ca-4799-b811-38b2c2cdf3a0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:18.485072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:18.486544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:18.487770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:18.489513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:18.515490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:18.522094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:18.565832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 879/1749] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:18.604139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:18.604291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193f238-636c-4bde-ac5c-dbe97ef65068 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:18.606591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:18.627879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:18.628868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:18.628868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:18.629050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] PATH_INFO: `/v3/users/791e44a94d8a4d64a5cd698c5758c493` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:18.631209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=791e44a94d8a4d64a5cd698c5758c493)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:18.633144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:18.637909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950889fb-59e0-4b34-bbfe-4397dea2d350 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:18.646171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:18.646993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:18.651698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:18.652525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:18.679949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:18.682089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ac44b-f63e-4073-80f3-3a486b2643d5 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:18.722840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 880/1750] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:18.841473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '791e44a94d8a4d64a5cd698c5758c493'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:18.879947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '791e44a94d8a4d64a5cd698c5758c493'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:18.891969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '791e44a94d8a4d64a5cd698c5758c493'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:18.892864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '791e44a94d8a4d64a5cd698c5758c493'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:18.894138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '791e44a94d8a4d64a5cd698c5758c493'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:18.985036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:19.018410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 871/1751] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:18 2022] DELETE /identity/v3/users/791e44a94d8a4d64a5cd698c5758c493 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:19.023115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724996a1-975d-4b98-b8d7-76a87c5278d6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:19.023405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724996a1-975d-4b98-b8d7-76a87c5278d6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:19.023680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724996a1-975d-4b98-b8d7-76a87c5278d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:19.090183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-724996a1-975d-4b98-b8d7-76a87c5278d6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:19.091438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-724996a1-975d-4b98-b8d7-76a87c5278d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:19.441379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 881/1752] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:19.710512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:19.712774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:20.114613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc36e4d-cbd2-4a9b-b8af-64264612f833 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:20.137009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:20.139076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:20.139689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:20.140907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:20.171791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:20.174400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5445c2-987b-42c3-a8e8-24b812fa385d service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:20.249269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 872/1753] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:21.792238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:21.792870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:21.792870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:21.794120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 882/1754] 10.222.0.49 () {54 vars in 912 bytes} [Wed Aug 10 11:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:13:21.806380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2055d-5bcb-42e6-b98f-799295d30e66 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:21.806984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2055d-5bcb-42e6-b98f-799295d30e66 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:21.806984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2055d-5bcb-42e6-b98f-799295d30e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:21.848258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f5a2055d-5bcb-42e6-b98f-799295d30e66 None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:21.849529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f5a2055d-5bcb-42e6-b98f-799295d30e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:21.879614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 873/1755] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:13:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:21.966739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:21.973488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:21.996287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512a813-424f-4f2f-8201-61c73aa24d17 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:22.000233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:22.000841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.001145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.001568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.022016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:22.023532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb2cb4-39a2-40c5-9548-637f37eb65a7 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:22.037473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 883/1756] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:22.271146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231e59-90ef-403f-8f21-1d720182b426 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.271630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231e59-90ef-403f-8f21-1d720182b426 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.271971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c231e59-90ef-403f-8f21-1d720182b426 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.274386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 874/1757] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:13:22.286644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.286972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.287203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.299877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:22.332305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:22.332991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:22.459438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11b9e-eb13-43ff-b77b-2cc2e33a4525 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.459916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11b9e-eb13-43ff-b77b-2cc2e33a4525 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.460332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e11b9e-eb13-43ff-b77b-2cc2e33a4525 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.462250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 875/1758] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:13:22.488994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.489792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.492862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.507570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 884/1759] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:13:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:22.535914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:22.537604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:22.553173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 876/1760] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:13:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:22.628961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:22.631544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:22.670603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891b5c9-6242-48ce-9a54-e25152daecef None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:22.693197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:22.696907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:22.698071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:22.698493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] PATH_INFO: `/v3/projects/9069fb8ae0da44e6af488a709f641ead` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:22.700445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9069fb8ae0da44e6af488a709f641ead)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:22.703687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:22.817362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:22.818946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:22.836547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9069fb8ae0da44e6af488a709f641ead', 'request_id': 'req-06d2988c-d88f-4e1d-85a8-cbc2930a0901'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:22.840508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ddd18-92dc-4517-8ab0-8ab7a202bd5e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:22.865999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9069fb8ae0da44e6af488a709f641ead', 'request_id': 'req-06d2988c-d88f-4e1d-85a8-cbc2930a0901'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:22.878000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9069fb8ae0da44e6af488a709f641ead', 'request_id': 'req-06d2988c-d88f-4e1d-85a8-cbc2930a0901'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:22.900725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] The token cache is being invalidate because project 9069fb8ae0da44e6af488a709f641ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:22.901113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:22.902313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:22.910786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 885/1761] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:22 2022] DELETE /identity/v3/projects/9069fb8ae0da44e6af488a709f641ead => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:22.910786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:22.913655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.015885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.016719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.017347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.017802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.024282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2988c-d88f-4e1d-85a8-cbc2930a0901 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.025814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.039618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.042208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.042701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.043225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.045524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.055252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.056909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.076345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.077990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39d13bb-3e3a-4dbf-aef6-c26a183d3f72 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.090765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 877/1762] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:23.093510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.097916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.104252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 886/1763] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:23.106460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547c8a-1993-460d-8b1c-5661dd6f03ec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.106897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547c8a-1993-460d-8b1c-5661dd6f03ec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.107303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49547c8a-1993-460d-8b1c-5661dd6f03ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.123534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:23.176743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-49547c8a-1993-460d-8b1c-5661dd6f03ec None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:23.177972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-49547c8a-1993-460d-8b1c-5661dd6f03ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.257573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeceb63c-7043-4a97-9d70-49581300d211 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.273053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.273631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.274795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.275003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] PATH_INFO: `/v3/users/2b994de001584ce2b98e3e10228a9718` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.276798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b994de001584ce2b98e3e10228a9718)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.278755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.293664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:23.303008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 887/1764] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:23.324411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.324848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.325130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.349129 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:23.350321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.363765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b994de001584ce2b98e3e10228a9718'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:23.373141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b994de001584ce2b98e3e10228a9718'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:23.374009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 888/1765] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:23.377967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.390075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36c681-7cd8-49cf-9f52-cfe9860c76c8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.394556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b994de001584ce2b98e3e10228a9718'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:23.395212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b994de001584ce2b98e3e10228a9718'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:23.395825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a04849ba-0b1c-45af-a3e4-cbf9f75786ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b994de001584ce2b98e3e10228a9718'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:23.405447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.407184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.407493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.408933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.409886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:23.410195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.415891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.423983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 878/1766] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:23 2022] DELETE /identity/v3/users/2b994de001584ce2b98e3e10228a9718 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:23.428654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.429377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.429994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.450966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 889/1767] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:23.453866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.456011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.473535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd17f6a-bc60-46a6-8211-bd9f1fbb2db3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.475453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:23.476708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.551725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.552326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.552603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.552877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.554635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.556690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.575529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 879/1768] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:23.589698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.591282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:23.601882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 890/1769] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:23.603314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.664170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec275-a258-44bf-9cd0-c4560ff2f9f9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.685620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1b8b6-60b9-41ee-898e-25952496c894 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.690741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.691503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.691858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.692201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.692921 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.694967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.713585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:23.718343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:23.723167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 891/1770] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:23.726393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.733048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.734836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.735142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.735421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.738234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4d0c52-f08e-452b-a243-0c5ba9c9aa9d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.744006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.745499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.745973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.746069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.746371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.747114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.747453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.748903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.757512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 880/1771] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:23.764511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 892/1772] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:23.765661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.776823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4000d-7b1e-4ded-84d3-d1a6ac0a2d4c service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.781701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.782309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.782618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.782901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.783683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.785282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb4acc6-1245-4c56-af7a-cd180f4fba49 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.796994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 881/1773] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:23.801373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:23.813309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac405e1-d413-40ef-ae41-12d515c43263 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:23.818656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:23.819402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.819773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.820120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] PATH_INFO: `/v3/projects/ff5f172ddd70468bbd05d15c08093e63/users/02f3ac99ac2d48269897df9b938ac830/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.840095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff5f172ddd70468bbd05d15c08093e63, user_id=02f3ac99ac2d48269897df9b938ac830, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:23.843453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:23.867514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 893/1774] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:23 2022] PUT /identity/v3/projects/ff5f172ddd70468bbd05d15c08093e63/users/02f3ac99ac2d48269897df9b938ac830/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:23.878827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:23.879324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:23.879859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:23.910038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] MFA Rules not processed for user `02f3ac99ac2d48269897df9b938ac830`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:23.911448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.018866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 882/1775] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:24.023176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.086367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f53aec-c9cc-4831-aeb4-0e2f9b447415 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.094746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.094970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.095372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.095877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.096281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.096469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.096917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.099227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.111835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ee9d4-3b34-435f-8ddb-d3d877fe5008 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.118309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.119175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.119446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.119770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] PATH_INFO: `/v3/projects/5544c57af7a24b389090f6537e3ae941` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.121517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5544c57af7a24b389090f6537e3ae941)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.123474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.131413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 894/1776] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:24 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:24.132378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.171353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5544c57af7a24b389090f6537e3ae941', 'request_id': 'req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.177167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5544c57af7a24b389090f6537e3ae941', 'request_id': 'req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.185013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5544c57af7a24b389090f6537e3ae941', 'request_id': 'req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.193612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096eb8-80d1-49d4-bf8a-453b0229a0ff admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.197060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] The token cache is being invalidate because project 5544c57af7a24b389090f6537e3ae941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:24.197385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.198040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.199623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 883/1777] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:24 2022] DELETE /identity/v3/projects/5544c57af7a24b389090f6537e3ae941 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:24.203355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.204650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.247992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:24.255805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:24.257312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.257934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.258463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.259271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.259908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.261941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.274164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f6c969-cd82-4dd7-a578-a0e79120a4d6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.280232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.280861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.281151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.281420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] PATH_INFO: `/v3/users/ecfb735713574d3882d6417d71f21cfc` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.282617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfb735713574d3882d6417d71f21cfc)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.284635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 895/1778] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:24 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:24.284749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.288437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.290003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.301309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d58ba7-d500-4296-963e-c0945bf9becf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.306189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.306812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.307082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.307350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.307982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.309568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.320428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 896/1779] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:24.325151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.332906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfb735713574d3882d6417d71f21cfc'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.335480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2e9e6-6836-456c-8cac-aed2e82b8280 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.340165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfb735713574d3882d6417d71f21cfc'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.341986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.342564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.342869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.343140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] PATH_INFO: `/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89/users/0bde5130a38e49828b74cdca1af5e7fd/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.348017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfb735713574d3882d6417d71f21cfc'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.348699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfb735713574d3882d6417d71f21cfc'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.349330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfb735713574d3882d6417d71f21cfc'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.358845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc711e2d507e4c928b3a8e45aecd2a89, user_id=0bde5130a38e49828b74cdca1af5e7fd, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.361653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.367613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 884/1780] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:24 2022] DELETE /identity/v3/users/ecfb735713574d3882d6417d71f21cfc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:24.377834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 897/1781] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:24 2022] PUT /identity/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89/users/0bde5130a38e49828b74cdca1af5e7fd/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:24.381694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.383166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.478505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55254f7f-e5b0-4aaa-8082-b627b9c39db7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.543180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.543813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.544099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.544378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.545565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.546792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d8c846-a5eb-4614-9fdb-d2b91ed21da8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.558138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 885/1782] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:24.562314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.574313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862f03a-ee45-480f-ba74-09d5cbd116d6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.580187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.580939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.581426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.581767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] PATH_INFO: `/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89/users/0bde5130a38e49828b74cdca1af5e7fd/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.588856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc711e2d507e4c928b3a8e45aecd2a89, user_id=0bde5130a38e49828b74cdca1af5e7fd, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.592373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.606279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 898/1783] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:24 2022] PUT /identity/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89/users/0bde5130a38e49828b74cdca1af5e7fd/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:24.616416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.616693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.616971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.637651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:24.648073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] MFA Rules not processed for user `0bde5130a38e49828b74cdca1af5e7fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:24.649312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.694590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.696711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.742186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 886/1784] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:24.753548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.755006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.787102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc874-2e80-40a3-9e2c-cc3f489995bf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.792243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.792869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.793152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.793445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] PATH_INFO: `/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.794566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3024da08d4a46aeb55cb2ca5b2277a1)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.796482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.811109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603814a-104e-4b22-ba98-4c8d23b8514e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:24.815863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:24.816526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.816832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.817293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.828221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:24.829813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7abe6f-5e09-4760-b789-772e8cb7dcdf service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:24.841250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 887/1785] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:24.850767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3024da08d4a46aeb55cb2ca5b2277a1', 'request_id': 'req-213cfffa-ddba-4b78-881b-1a0015ceac6f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.858168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3024da08d4a46aeb55cb2ca5b2277a1', 'request_id': 'req-213cfffa-ddba-4b78-881b-1a0015ceac6f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.870076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3024da08d4a46aeb55cb2ca5b2277a1', 'request_id': 'req-213cfffa-ddba-4b78-881b-1a0015ceac6f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.882453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] The token cache is being invalidate because project d3024da08d4a46aeb55cb2ca5b2277a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:24.882880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.883627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:24.885295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 899/1786] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:24 2022] DELETE /identity/v3/projects/d3024da08d4a46aeb55cb2ca5b2277a1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:24.891677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1905-d40a-4812-8a7a-738742422094 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:24.892153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1905-d40a-4812-8a7a-738742422094 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:24.893253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1905-d40a-4812-8a7a-738742422094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:24.918188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-7bcc1905-d40a-4812-8a7a-738742422094 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:24.919480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7bcc1905-d40a-4812-8a7a-738742422094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.925911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:24.927388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:24.963212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:24.994621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 888/1787] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:25.002678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.002963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.003436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.004128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cfffa-ddba-4b78-881b-1a0015ceac6f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.008913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.009711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.009997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.010348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.012635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:25.030303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:25.032438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:25.056866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 889/1788] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:25.058372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.070412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2fbc5-479c-458e-bf28-f164486f152b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.076887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.077894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.078274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.078685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.079423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:25.079817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.081591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.099854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 890/1789] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:25.101946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.103542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.105361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.120501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ca99e-a2c4-4e2c-a322-b84b0be0f3ab admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.125031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 900/1790] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:25.127460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.128080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.128363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.128637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.129270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.131419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.147827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:25.164180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 891/1791] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/projects => generated 407 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:25.168831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.183217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aca4e6-bbd1-4190-bdda-ce2988635a68 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.258506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.259876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.261433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.262433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.264168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.266718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.289323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 901/1792] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/users => generated 304 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:25.293765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.303937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebb0bd-1db2-4f36-aec8-4e0efcca31a7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.309335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.309958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.310242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.310518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.311196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.312798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.323339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 892/1793] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:25.327211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.337062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082108f4-f802-491e-8d99-3c09eaa36d75 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.341657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.342245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.342549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.342852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.343490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.345104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2becf476-91f9-4da7-b165-f9c78fd9284b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.355148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 902/1794] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:25.359034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.369192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b672fa5-8835-4182-9ff8-b8b6b9f2cfd4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.373785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.374422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.374745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.375022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] PATH_INFO: `/v3/projects/711deccd36084bb7b7ac7c826c8d299c/users/76059462afde40818cb932abfcfb4fcf/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.389857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] RBAC: Authorizing `identity:create_grant(project_id=711deccd36084bb7b7ac7c826c8d299c, user_id=76059462afde40818cb932abfcfb4fcf, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.392559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.405316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 893/1795] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:25 2022] PUT /identity/v3/projects/711deccd36084bb7b7ac7c826c8d299c/users/76059462afde40818cb932abfcfb4fcf/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:25.415564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.415851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.416127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.444546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] MFA Rules not processed for user `76059462afde40818cb932abfcfb4fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:25.445824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:25.493836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:25.513226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 903/1796] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:25.517739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.580765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ad7a0-31cc-4f02-96f4-b594d72a66de admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.587357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.587992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.588271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.588569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.589201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.591105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.624389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 894/1797] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/projects => generated 399 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:25.628358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.640369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cf1a3-70a3-458d-8684-5aa9e821b96b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.698338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.698983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.699258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.699527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.700145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.702052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.722605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 904/1798] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/users => generated 306 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:25.726357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.736934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c2ece-b3aa-4ff1-a2b6-6a9db0e3cdb7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.741470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.742049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.742332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.742638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.743926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.745505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.755710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 895/1799] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:25.759740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.769401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48284527-edc2-4ef5-b15a-0580232690f0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.773956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.774691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.774974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.775247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] PATH_INFO: `/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b/users/423f8be4c4784e7eaa9bf5d4f97a4085/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.789367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b62f5fbd9245a1a8e2ca243404601b, user_id=423f8be4c4784e7eaa9bf5d4f97a4085, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.792577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.805170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 905/1800] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:25 2022] PUT /identity/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b/users/423f8be4c4784e7eaa9bf5d4f97a4085/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:25.809609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.819538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bbbae0-f43c-4cd8-afa5-6cf4b7913446 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.874401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.875108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.875388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.875660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.876297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.877919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e4cdfd-ea89-46f8-848d-735ee0abe180 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.887415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 896/1801] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:25.891646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:25.902400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6101658-6eca-4bae-8bd1-0d5c87c06e6b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:25.906981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:25.907622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.907903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.908175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] PATH_INFO: `/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b/users/423f8be4c4784e7eaa9bf5d4f97a4085/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.910074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1b62f5fbd9245a1a8e2ca243404601b, user_id=423f8be4c4784e7eaa9bf5d4f97a4085, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:25.912700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:25.923231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 906/1802] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:25 2022] PUT /identity/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b/users/423f8be4c4784e7eaa9bf5d4f97a4085/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:25.933503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:25.934190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:25.934190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:25.960762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] MFA Rules not processed for user `423f8be4c4784e7eaa9bf5d4f97a4085`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:25.962489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:25.981837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:26.034297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 897/1803] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:26.045621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:26.047566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:26.116893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85752e-855c-4200-a4ee-4971ac1c7d7b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:26.122352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:26.123244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:26.123692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:26.124101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:26.136487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:26.137924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:26.149008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 907/1804] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:26 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:26.389250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:26.403424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d257b-5bcf-47a3-a08c-32daf47d10cc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:26.407860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:26.408605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:26.408984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:26.409350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:26.511284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:26.513006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9f513-d966-49fb-9572-844aa2aa3031 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:26.525960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 898/1805] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:26 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:36.144241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:36.519826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=48 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:46.823060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:46.908038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c223f-20d6-436a-9337-9eaaf00c950e service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:46.912506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:46.913128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:46.913408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:46.913679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] PATH_INFO: `/v3/users/76059462afde40818cb932abfcfb4fcf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:46.914860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76059462afde40818cb932abfcfb4fcf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:46.916744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.001136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76059462afde40818cb932abfcfb4fcf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.009487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76059462afde40818cb932abfcfb4fcf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.027814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76059462afde40818cb932abfcfb4fcf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.027814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76059462afde40818cb932abfcfb4fcf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.027814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76059462afde40818cb932abfcfb4fcf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.050333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 908/1806] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:46 2022] DELETE /identity/v3/users/76059462afde40818cb932abfcfb4fcf => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:47.055312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.055793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.056084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.088295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:47.089708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.177870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.179675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.198033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 899/1807] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:47.211408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.214603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.305859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcde941-fec7-4345-85e1-a8a370316cfa None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:47.310487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:47.311557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.312197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.312735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.318919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957508f-385b-4f21-9200-3f4004f63c3b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:47.324363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:47.325665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:47.326685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.326967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.327087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d67cd1-1118-44df-8067-42a0b9260d95 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.327283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] PATH_INFO: `/v3/users/02f3ac99ac2d48269897df9b938ac830` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.328415 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=02f3ac99ac2d48269897df9b938ac830)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:47.330322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.337915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 900/1808] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:47.358136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:47.389418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02f3ac99ac2d48269897df9b938ac830'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.398531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02f3ac99ac2d48269897df9b938ac830'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.407694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02f3ac99ac2d48269897df9b938ac830'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.408609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02f3ac99ac2d48269897df9b938ac830'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.410717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02f3ac99ac2d48269897df9b938ac830'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.442493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.442806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.443086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.446688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 909/1809] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:47 2022] DELETE /identity/v3/users/02f3ac99ac2d48269897df9b938ac830 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:47.463450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:47.464730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.549142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 901/1810] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:47.558947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.560465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.620602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2331990-1511-40f6-bc4c-085821c9ef1c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:47.624841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:47.625602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.625879 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.626186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.636522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.637930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:47.638387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.639469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.652760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a3081-ad6a-46ec-8586-21a150a31aa3 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:47.656768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 910/1811] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:47.660577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:47.661247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.661532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.661842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] PATH_INFO: `/v3/projects/711deccd36084bb7b7ac7c826c8d299c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.663140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] RBAC: Authorizing `identity:delete_project(project_id=711deccd36084bb7b7ac7c826c8d299c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:47.665424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.708482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:47.717378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711deccd36084bb7b7ac7c826c8d299c', 'request_id': 'req-706c59b3-dd59-4093-b493-d88a5d7a326b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.723635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711deccd36084bb7b7ac7c826c8d299c', 'request_id': 'req-706c59b3-dd59-4093-b493-d88a5d7a326b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.731647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711deccd36084bb7b7ac7c826c8d299c', 'request_id': 'req-706c59b3-dd59-4093-b493-d88a5d7a326b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.743217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] The token cache is being invalidate because project 711deccd36084bb7b7ac7c826c8d299c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:47.743558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.744218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.745808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 902/1812] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:47 2022] DELETE /identity/v3/projects/711deccd36084bb7b7ac7c826c8d299c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:47.749898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.759899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d914b0f-7112-4aa1-be34-227a4fd66109 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:47.823580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:47.824197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:47.824197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:47.824465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] PATH_INFO: `/v3/users/423f8be4c4784e7eaa9bf5d4f97a4085` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:47.825606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=423f8be4c4784e7eaa9bf5d4f97a4085)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:47.827562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:47.887531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:47.889823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423f8be4c4784e7eaa9bf5d4f97a4085'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.905495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423f8be4c4784e7eaa9bf5d4f97a4085'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.913789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423f8be4c4784e7eaa9bf5d4f97a4085'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.914659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423f8be4c4784e7eaa9bf5d4f97a4085'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.915286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423f8be4c4784e7eaa9bf5d4f97a4085'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:47.945275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 911/1813] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:47 2022] DELETE /identity/v3/users/423f8be4c4784e7eaa9bf5d4f97a4085 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:47.946969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:47.948940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:47.972672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c59b3-dd59-4093-b493-d88a5d7a326b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:48.079983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:48.080609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.080808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.081083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] PATH_INFO: `/v3/projects/ff5f172ddd70468bbd05d15c08093e63` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.082464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff5f172ddd70468bbd05d15c08093e63)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:48.084419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:48.134386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff5f172ddd70468bbd05d15c08093e63', 'request_id': 'req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.140456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff5f172ddd70468bbd05d15c08093e63', 'request_id': 'req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.148325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff5f172ddd70468bbd05d15c08093e63', 'request_id': 'req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.160578 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] The token cache is being invalidate because project ff5f172ddd70468bbd05d15c08093e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:48.160873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.161533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.163071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 903/1814] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:47 2022] DELETE /identity/v3/projects/ff5f172ddd70468bbd05d15c08093e63 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:48.167817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:48.169368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.183729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7167b-886f-4a4d-a899-beeeef4e7104 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:48.248119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:48.248754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.249017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.249310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] PATH_INFO: `/v3/users/0bde5130a38e49828b74cdca1af5e7fd` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.251417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bde5130a38e49828b74cdca1af5e7fd)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:48.253421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:48.317576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bde5130a38e49828b74cdca1af5e7fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.327185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bde5130a38e49828b74cdca1af5e7fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.342508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bde5130a38e49828b74cdca1af5e7fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.343238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bde5130a38e49828b74cdca1af5e7fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.343934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-49dbf1c7-6ff2-431a-af4b-076371802bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bde5130a38e49828b74cdca1af5e7fd'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.374001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 912/1815] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:48 2022] DELETE /identity/v3/users/0bde5130a38e49828b74cdca1af5e7fd => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:48.497361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:48.498912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.522711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb3e04e-c8ba-4bfa-99b4-b87f5d80dbe5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:48.644449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:48.656334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:48.657933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.660510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.660906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] PATH_INFO: `/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.662798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1b62f5fbd9245a1a8e2ca243404601b)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:48.665498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:48.723524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1b62f5fbd9245a1a8e2ca243404601b', 'request_id': 'req-159794fb-ed0a-4c2e-84ba-79773d9787b5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.747523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1b62f5fbd9245a1a8e2ca243404601b', 'request_id': 'req-159794fb-ed0a-4c2e-84ba-79773d9787b5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.759795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1b62f5fbd9245a1a8e2ca243404601b', 'request_id': 'req-159794fb-ed0a-4c2e-84ba-79773d9787b5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.781213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] The token cache is being invalidate because project c1b62f5fbd9245a1a8e2ca243404601b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:48.781664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.783306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-159794fb-ed0a-4c2e-84ba-79773d9787b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:48.788656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 904/1816] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:48 2022] DELETE /identity/v3/projects/c1b62f5fbd9245a1a8e2ca243404601b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:48.797824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09da9-f30f-4469-96f8-3e875186c0b6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.798252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09da9-f30f-4469-96f8-3e875186c0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.798767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b09da9-f30f-4469-96f8-3e875186c0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.820941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-09b09da9-f30f-4469-96f8-3e875186c0b6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:48.822513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-09b09da9-f30f-4469-96f8-3e875186c0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.840434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811c38-1f54-483d-a566-4cf5109cfc22 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.840966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811c38-1f54-483d-a566-4cf5109cfc22 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.841245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c811c38-1f54-483d-a566-4cf5109cfc22 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.845077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 905/1817] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:13:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:13:48.857833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.858531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.859387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.872691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:48.891310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:48.893610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.911638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 913/1818] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:48.920452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.920750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.921034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.952510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:48.953702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.964051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 914/1819] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:48.965204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:48.966816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:48.980616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1544c-c7f5-4d28-ba5d-22b8af2b07da None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:48.985835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:48.986699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:48.986984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:48.987274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] PATH_INFO: `/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:48.988378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc711e2d507e4c928b3a8e45aecd2a89)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:48.990270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:48.992917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 906/1820] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:48.993840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.005093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845a8e9-9282-45dc-85f3-863b1225730d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.010287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.010955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.011269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.011546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.012208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:49.012513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.014098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.026719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 907/1821] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:49.027726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.029607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.043140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc711e2d507e4c928b3a8e45aecd2a89', 'request_id': 'req-864ba18e-5d0e-4aca-a682-c9792e526c21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.045999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcbf34-f25a-469c-ae2c-9694b47fb0b4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.048616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc711e2d507e4c928b3a8e45aecd2a89', 'request_id': 'req-864ba18e-5d0e-4aca-a682-c9792e526c21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.056214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc711e2d507e4c928b3a8e45aecd2a89', 'request_id': 'req-864ba18e-5d0e-4aca-a682-c9792e526c21'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.069275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] The token cache is being invalidate because project fc711e2d507e4c928b3a8e45aecd2a89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:49.069596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.070259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.074994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 915/1822] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:48 2022] DELETE /identity/v3/projects/fc711e2d507e4c928b3a8e45aecd2a89 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:49.074994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.074994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.106776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:49.117122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.118340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.118877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.119331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.121959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.133963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.135570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ea0f8-f83a-408a-bfbb-1e95946e10cd service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.141264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ba18e-5d0e-4aca-a682-c9792e526c21 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.145622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 908/1823] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:49.145996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.146630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.146813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.147024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.147131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.147359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.147457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.148019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.149906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.168938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:49.170385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.189803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 909/1824] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:49.189803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.190486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 916/1825] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:49.190486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.191941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.203541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7765c91-ad80-4095-96c8-b9494512ad6f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.236564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:49.257089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5143f12-9bcd-41ab-8819-ac269afbbea3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.261686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.262301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.262412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.262601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.262898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.263062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.263341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.263528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.263621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] PATH_INFO: `/v3/users/e01531151f024622b7dae0dcb23aa8f4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.264768 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e01531151f024622b7dae0dcb23aa8f4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.265559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ed96-848f-4bb1-9754-17a6c526ceb1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.266724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.287410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 917/1826] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:49.288645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.288930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.289205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.311110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:49.312302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.316762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e01531151f024622b7dae0dcb23aa8f4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.320576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 918/1827] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:49.321242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.323904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e01531151f024622b7dae0dcb23aa8f4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.331191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e01531151f024622b7dae0dcb23aa8f4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.331813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e01531151f024622b7dae0dcb23aa8f4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.332428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e01531151f024622b7dae0dcb23aa8f4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:49.334329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07837b2-9d6e-4db5-b08a-861b728c9f00 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.340360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.341340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.341340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.341500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.342126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.343776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2637a6-d203-4d7d-a208-768151dc0071 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.352194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 910/1828] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:49 2022] DELETE /identity/v3/users/e01531151f024622b7dae0dcb23aa8f4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:49.353008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.357682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 919/1829] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:49.358059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.358355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.358663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.359357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.382134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:49.383895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.445893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c8068-03c4-4894-ada5-d1277791e4fb admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.450829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.451446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.451731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.452804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.452804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:49.452980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.456713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 920/1830] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:49.456713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.456939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.457451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.468480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0457cfc-4d44-4310-86ba-4789d1313892 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.468916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:49.480148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 911/1831] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:49.481044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.481350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.482001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.482279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.482547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.483310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.484895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.485183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.495885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 921/1832] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:49.496692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.506676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706644-202d-4267-a868-c06cad42b547 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.512349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.512948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.513233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.513513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.514134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.516698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.554820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 922/1833] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:49.555750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.559486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002eabae-ce87-4146-8a95-9836699bd05b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.617466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.618011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.618293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.618565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.644634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc7096-d993-4c46-9a91-c891f4445d2d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.650002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.650802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.651085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.651392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] PATH_INFO: `/v3/projects/604ff564e09641e9b4f3921e2fde2ab4/users/64a1b90d4c004a55ac1a213060b2bb83/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.666866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=604ff564e09641e9b4f3921e2fde2ab4, user_id=64a1b90d4c004a55ac1a213060b2bb83, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.669994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.681057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.682751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c73dc5-3923-41a9-88c6-a66857010325 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.683628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 923/1834] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:49 2022] PUT /identity/v3/projects/604ff564e09641e9b4f3921e2fde2ab4/users/64a1b90d4c004a55ac1a213060b2bb83/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:49.684438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.689666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:49.696803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 912/1835] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:49.698018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.698872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.699165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.728562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] MFA Rules not processed for user `64a1b90d4c004a55ac1a213060b2bb83`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:49.729765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:49.764275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98c91b-73bc-4655-9a49-265607f86fa8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.772042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.773111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.773612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.774739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.776176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.780061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.804935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:49.816049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 924/1836] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:49.817579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.829637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 913/1837] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:49.834890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.845655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968b726-6538-4335-93c3-4936e893ff38 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.858856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18eb992-edb9-44c5-af8f-2c0c9b447732 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.866315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.867233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.867635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.867788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.868466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.870667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.877002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.877832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.878290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.878971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.879935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.882610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 925/1838] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:49.884956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.887567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.900033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a0fc5-e1e5-4bd3-8102-b784757be1a5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:49.906031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:49.907596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:49.908166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:49.908656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:49.909817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:49.912315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:49.924789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 914/1839] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/projects => generated 367 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:49.926099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.933764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 926/1840] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:49.935643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:49.968091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ce65b0-ff10-4b7e-9c13-414e87276d04 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.071482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9053a41a-4c62-447c-b88c-bd45b1e763ed admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.076995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.077552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.078077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.078150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.092584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.094710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.096470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.097639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.098104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.098360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] PATH_INFO: `/v3/projects/3ec61071cdaf4bff9659c4c163062931/users/83ed2d36788d4cc1868ba6124a54cdac/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.118881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec61071cdaf4bff9659c4c163062931, user_id=83ed2d36788d4cc1868ba6124a54cdac, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.122450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fddbcf0-57ba-4b94-ad20-cb954cdc3600 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.123558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 915/1841] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:49 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:50.127523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.139386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76954ba-66be-4e73-b4cb-979400036137 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.144010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 927/1842] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:49 2022] PUT /identity/v3/projects/3ec61071cdaf4bff9659c4c163062931/users/83ed2d36788d4cc1868ba6124a54cdac/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:50.155474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.155716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.156275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.221158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] MFA Rules not processed for user `83ed2d36788d4cc1868ba6124a54cdac`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:50.222744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.232315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.232942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.233265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.233501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.234358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.235900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.247753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 916/1843] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:50.252097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.253809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.271978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075f892-6323-485a-98bc-bfd33bb4cf59 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.278431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.279123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.279774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.280946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] PATH_INFO: `/v3/projects/a4b7e68000ed472c88481b04ad050898` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.281303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4b7e68000ed472c88481b04ad050898)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.281828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:50.283303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.317013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 928/1844] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:50.317903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.330828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8448834-f8b0-4950-ac06-a9641929c474 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.344296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4b7e68000ed472c88481b04ad050898', 'request_id': 'req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.347259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:50.353264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4b7e68000ed472c88481b04ad050898', 'request_id': 'req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.363029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4b7e68000ed472c88481b04ad050898', 'request_id': 'req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.377196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] The token cache is being invalidate because project a4b7e68000ed472c88481b04ad050898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:50.377636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.378480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.380226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 917/1845] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:50 2022] DELETE /identity/v3/projects/a4b7e68000ed472c88481b04ad050898 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:50.381092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.383253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.402361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.402984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.403271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.403546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] PATH_INFO: `/v3/projects/b224f8048c274d5187f1b7b00f99542d/users/16b0bf813e2b46f182b59a864e054dee/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.420783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b224f8048c274d5187f1b7b00f99542d, user_id=16b0bf813e2b46f182b59a864e054dee, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.423473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.438489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 929/1846] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:50 2022] PUT /identity/v3/projects/b224f8048c274d5187f1b7b00f99542d/users/16b0bf813e2b46f182b59a864e054dee/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:50.439566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.441273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.461526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d309e8-cf56-4f8b-9a6b-53ca6f19cf13 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.517379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd490ac7-ad89-4798-a978-367f4b934521 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.522338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.523591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.523591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.523591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] PATH_INFO: `/v3/users/68857b40776f431e8ec0bcc0f442a43f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.524753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68857b40776f431e8ec0bcc0f442a43f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.527248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.529625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.530242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.530698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.530981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.531625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.533661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.568575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 918/1847] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:50 2022] POST /identity/v3/projects => generated 373 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:50.569438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.572907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.584001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68857b40776f431e8ec0bcc0f442a43f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.591925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68857b40776f431e8ec0bcc0f442a43f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.600779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68857b40776f431e8ec0bcc0f442a43f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.601775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68857b40776f431e8ec0bcc0f442a43f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.602970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68857b40776f431e8ec0bcc0f442a43f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:50.627661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:50.665975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 930/1848] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:50 2022] DELETE /identity/v3/users/68857b40776f431e8ec0bcc0f442a43f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:50.668318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.675235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:50.741254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b738882-7017-41ef-8ee3-09aa8bbfdec5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.783948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca79395-6f61-4283-873c-1b0ffd91064a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.791642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.792405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.792753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.793095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.795270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.798157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.800726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:50.857002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 931/1849] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:50 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:50.862847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.883892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19d885-e07c-4db1-8716-c17853058da2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.893348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.894161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.894536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.894848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.895592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.898432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.914712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 932/1850] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:50.918593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.939196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64474910-b496-4f1b-9589-21e9f43e484e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:50.949691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.952324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:50.952880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.953267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.953552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] PATH_INFO: `/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9/users/aa498a1b4cea418db8c76c6dd560ba11/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.953673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:50.953868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:50.954151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:50.955429 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.960880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.985530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae3f205ed7d4594a41f1f09b61f6da9, user_id=aa498a1b4cea418db8c76c6dd560ba11, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:50.989878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 919/1851] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:50.992933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:50.993770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:50.996780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.023878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 933/1852] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:50 2022] PUT /identity/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9/users/aa498a1b4cea418db8c76c6dd560ba11/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:51.032523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656284f3-8c91-4352-880d-4d9be9e9e7ca admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.047644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.108509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.109426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.110227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.110227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] PATH_INFO: `/v3/projects/b224f8048c274d5187f1b7b00f99542d/users/16b0bf813e2b46f182b59a864e054dee/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.113607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b224f8048c274d5187f1b7b00f99542d, user_id=16b0bf813e2b46f182b59a864e054dee, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.117062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa11bc9-5756-429b-b799-5d88f22d60c5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.130639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 920/1853] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:50 2022] PUT /identity/v3/projects/b224f8048c274d5187f1b7b00f99542d/users/16b0bf813e2b46f182b59a864e054dee/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:51.136847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49267de-b7e1-4572-a85f-f804794ffb27 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.143411 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.144415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.144537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.207360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:51.221154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] MFA Rules not processed for user `16b0bf813e2b46f182b59a864e054dee`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:51.226965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.231974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.233823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.234235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.234795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.235690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.238077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.255723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 934/1854] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:51.261381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.294927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634365a-72d6-4193-bd16-970ce73beea2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.308474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.310715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.310715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.310715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] PATH_INFO: `/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9/users/aa498a1b4cea418db8c76c6dd560ba11/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.313359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae3f205ed7d4594a41f1f09b61f6da9, user_id=aa498a1b4cea418db8c76c6dd560ba11, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.316976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2459ba1-b278-4dc7-9456-6238ab786b66 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.355391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 935/1855] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:51 2022] PUT /identity/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9/users/aa498a1b4cea418db8c76c6dd560ba11/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:51.366243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.366612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.367128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.430499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] MFA Rules not processed for user `aa498a1b4cea418db8c76c6dd560ba11`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:51.431807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.453274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 921/1856] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:51.468089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.470879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.536984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:51.574458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef8809-163e-4080-8cb7-ab555a74e316 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.675848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.676727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.677148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.677588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.681578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 936/1857] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:51.704928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.706013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.721658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6ecc-55b6-4e01-8b16-2d6bd80887ee None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.742398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.743680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.744944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.745936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.771970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.774618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.800732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 937/1858] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:51.800732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.803151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:51.804307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.806062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.833403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 922/1859] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:51.892556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e4e80f-2ea6-4296-b14e-f9396237b657 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.899539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:51.900184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:51.900460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:51.900752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] PATH_INFO: `/v3/projects/355d75e2950240dd9a00ac915c202a56` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:51.902874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=355d75e2950240dd9a00ac915c202a56)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:51.905433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:51.955579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:51.959640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355d75e2950240dd9a00ac915c202a56', 'request_id': 'req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:51.968472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355d75e2950240dd9a00ac915c202a56', 'request_id': 'req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:51.979500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:51.980703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355d75e2950240dd9a00ac915c202a56', 'request_id': 'req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:51.988607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6606455-e35d-4054-b02a-a4254b029839 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:51.992986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] The token cache is being invalidate because project 355d75e2950240dd9a00ac915c202a56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:51.993283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:51.994059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f72d5762-3fcd-489d-a2dd-4ce519ce4e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:51.995692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 938/1860] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:51 2022] DELETE /identity/v3/projects/355d75e2950240dd9a00ac915c202a56 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:52.000739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c39bfd-f9df-466b-b393-4bb90b149566 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.001029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c39bfd-f9df-466b-b393-4bb90b149566 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.001310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c39bfd-f9df-466b-b393-4bb90b149566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.024346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-45c39bfd-f9df-466b-b393-4bb90b149566 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:52.025558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-45c39bfd-f9df-466b-b393-4bb90b149566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.057909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.058720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.059102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.062972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.065073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.103384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 939/1861] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:52.111295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.111577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.112998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.139066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:52.140383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.145463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:52.148935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.151250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.162981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 940/1862] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:52.163965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.169231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 923/1863] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:52.178365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1919ee14-870a-4e44-9238-61f6b5a4ddd8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.184281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.185009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.185295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.185563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.186215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:52.186602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.188167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.204618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 941/1864] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:52.205933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.218846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ad217-eb05-46ce-92ba-ea7ee7df3736 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.224354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.225398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.225799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.226150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.227230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.230839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.240894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.242616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.269968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbce1b3-07cd-4682-942d-26397d60693b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.278829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.279485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.279910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.280194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] PATH_INFO: `/v3/users/3acb858957604c7b8cf1789d330e59d4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.281392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3acb858957604c7b8cf1789d330e59d4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.283581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.309565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 924/1865] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:52 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:52.317898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.405949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3acb858957604c7b8cf1789d330e59d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.415595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3acb858957604c7b8cf1789d330e59d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.431722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3acb858957604c7b8cf1789d330e59d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.433429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3acb858957604c7b8cf1789d330e59d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.434365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e692956-9a30-420b-b527-b0710bce8665 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.437052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c30bb87d-75f5-4e44-9e24-537cf339871f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3acb858957604c7b8cf1789d330e59d4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.440837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.441557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.441958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.442342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.443271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.445850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.478011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 925/1866] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:52 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:52.480114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.487939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.495887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 942/1867] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:52 2022] DELETE /identity/v3/users/3acb858957604c7b8cf1789d330e59d4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:52.497794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.499261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.499864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.520368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b95bb-e2df-4af1-8005-213a6e0326d4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.546891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:52.548439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.606637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:52.644468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.645359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.645717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.646119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.647055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.649027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.668230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 926/1868] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:52.668230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.669767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.688031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 943/1869] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:52.689794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.703192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ea9002-dcf8-4a67-9fe5-a25776ca31cf admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.705048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bdfa7-657c-4431-858e-a83eca0d48b7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.710950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.711739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.711848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.712225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.712746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.715078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.742259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 944/1870] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:52.742259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.742480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.742480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.742480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.742480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] PATH_INFO: `/v3/users/83ed2d36788d4cc1868ba6124a54cdac` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.742844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83ed2d36788d4cc1868ba6124a54cdac)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.742927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.743008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.823035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83ed2d36788d4cc1868ba6124a54cdac'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.840030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:52.842461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d118d78-7eb5-49d2-80b1-375c47ea1699 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.845267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83ed2d36788d4cc1868ba6124a54cdac'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.867013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:52.868226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.869264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83ed2d36788d4cc1868ba6124a54cdac'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.869654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.870104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83ed2d36788d4cc1868ba6124a54cdac'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.870516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.870843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a36cfbe4-cb43-41b3-b78e-8caef94dfde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83ed2d36788d4cc1868ba6124a54cdac'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:52.875992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.893295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:52.894756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:52.906344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 945/1871] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:52.907508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 927/1872] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:52 2022] DELETE /identity/v3/users/83ed2d36788d4cc1868ba6124a54cdac => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:52.908380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:52.911744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:52.913489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff736b-73bf-43e9-9ee2-da1a820f9eea None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:52.913923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff736b-73bf-43e9-9ee2-da1a820f9eea None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:52.914339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff736b-73bf-43e9-9ee2-da1a820f9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:52.927739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9edf0-7664-4a14-a487-43198c6ce4d9 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:52.944940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-09ff736b-73bf-43e9-9ee2-da1a820f9eea None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:52.947369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-09ff736b-73bf-43e9-9ee2-da1a820f9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.034863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.036662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.037080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.037351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] PATH_INFO: `/v3/projects/9f3781c122ed429aaaf93f42d43180e0/users/920e1914cddd453d9654246158367c3c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.051499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 928/1873] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:53.063765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3781c122ed429aaaf93f42d43180e0, user_id=920e1914cddd453d9654246158367c3c, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.066811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.093139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 946/1874] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:52 2022] PUT /identity/v3/projects/9f3781c122ed429aaaf93f42d43180e0/users/920e1914cddd453d9654246158367c3c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:53.103716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.106586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.106796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.107085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.107464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.207267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:53.212225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] MFA Rules not processed for user `920e1914cddd453d9654246158367c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:53.219406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.232378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbbb04-0b74-47a3-8a88-4108e1799b6d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.237542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.238415 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.238901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.239382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.328273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.329646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.345402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 947/1875] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:53.413306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 929/1876] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:53.417713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.434938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e9008-d29e-4a2d-aa92-32209cf3c57b service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.442309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.443468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.443817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.444141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.444834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.447690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.492146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 948/1877] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:53 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:53.493381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.504630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dc041-652e-4c19-b769-8bbaf57cfa32 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.574635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.575418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.575530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.575791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.576423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.578430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.583193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.585104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.594510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:53.601916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe1724-9599-4e56-8ec7-907d1cf7a4fd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.607565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.608261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.608614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.609068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] PATH_INFO: `/v3/projects/801934b94e6843169862f95a88203d0c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.610376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=801934b94e6843169862f95a88203d0c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.612486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.616796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 930/1878] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:53 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:53.619120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.650915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486baf5-eefa-4ab2-bd0e-e4579d7a27f7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.671191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.671845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.672129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.672404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.673096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.674703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.690671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 931/1879] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:53.694433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.707448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d7775b-6b69-4fab-950a-a735e8482996 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.735407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '801934b94e6843169862f95a88203d0c', 'request_id': 'req-abf18725-244b-4bc3-9a05-f9ef5169a201'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.751995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '801934b94e6843169862f95a88203d0c', 'request_id': 'req-abf18725-244b-4bc3-9a05-f9ef5169a201'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.766813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '801934b94e6843169862f95a88203d0c', 'request_id': 'req-abf18725-244b-4bc3-9a05-f9ef5169a201'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.783097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.784248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.784854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.785412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.786213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.788241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.791209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] The token cache is being invalidate because project 801934b94e6843169862f95a88203d0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:53.791534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.802601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.805113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 949/1880] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:53 2022] DELETE /identity/v3/projects/801934b94e6843169862f95a88203d0c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:53.810118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.811577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.813473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 932/1881] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:53.814362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:53.816058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:53.829212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf18725-244b-4bc3-9a05-f9ef5169a201 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.835408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.836190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.836544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.836891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] PATH_INFO: `/v3/users/92b66460ba6549bfbf71852953bcc9a7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.838393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b66460ba6549bfbf71852953bcc9a7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.841195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.842273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7517e-b190-465a-adf7-c1b0c6c22ebf admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:53.850526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:53.851233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.851523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.851803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] PATH_INFO: `/v3/projects/cad42fb175464edea1f2145e2ff4b9e0/users/f3a032e7b06042688389c1cb4da511db/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.889274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad42fb175464edea1f2145e2ff4b9e0, user_id=f3a032e7b06042688389c1cb4da511db, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:53.893236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01efd0-e199-48bc-bafe-93347edba62d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:53.910892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 933/1882] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:53 2022] PUT /identity/v3/projects/cad42fb175464edea1f2145e2ff4b9e0/users/f3a032e7b06042688389c1cb4da511db/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:53.923681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:53.923986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:53.925084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:53.931111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b66460ba6549bfbf71852953bcc9a7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.933867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=84,keystone:INSERT=2,keystone:DELETE=14 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:53.948745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b66460ba6549bfbf71852953bcc9a7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.964313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b66460ba6549bfbf71852953bcc9a7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.965070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b66460ba6549bfbf71852953bcc9a7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.967711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b66460ba6549bfbf71852953bcc9a7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:53.976459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] MFA Rules not processed for user `f3a032e7b06042688389c1cb4da511db`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:53.978414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.002674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 950/1883] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:53 2022] DELETE /identity/v3/users/92b66460ba6549bfbf71852953bcc9a7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:54.046726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.048909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.111433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 934/1884] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:54.112942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.115120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.207138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5780a-9943-48cd-9479-5c7b7c53bd14 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.215269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:54.223889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.224149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.224414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.224765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] PATH_INFO: `/v3/projects/3ec61071cdaf4bff9659c4c163062931` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.226029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec61071cdaf4bff9659c4c163062931)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.234355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.236769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9995f-085a-441a-af28-b8020d3fa75a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.242994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.243758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.244899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.245789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.246970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.250010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.290308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec61071cdaf4bff9659c4c163062931', 'request_id': 'req-fe959134-5fd8-4e6e-b47e-1778a9d974f4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.296695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec61071cdaf4bff9659c4c163062931', 'request_id': 'req-fe959134-5fd8-4e6e-b47e-1778a9d974f4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.302743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 935/1885] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:54.308112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec61071cdaf4bff9659c4c163062931', 'request_id': 'req-fe959134-5fd8-4e6e-b47e-1778a9d974f4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.308288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.323743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] The token cache is being invalidate because project 3ec61071cdaf4bff9659c4c163062931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:54.324042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.324832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.330344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 951/1886] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:54 2022] DELETE /identity/v3/projects/3ec61071cdaf4bff9659c4c163062931 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:54.330863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.332797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.399530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:54.408166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe959134-5fd8-4e6e-b47e-1778a9d974f4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.409127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abeeb3-1bd4-4018-b5e7-c70e6570a46b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.415242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.415732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.416074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.416360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.416472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.416643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.416895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] PATH_INFO: `/v3/users/aa498a1b4cea418db8c76c6dd560ba11` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.417013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.417573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.418489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa498a1b4cea418db8c76c6dd560ba11)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.419773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.420645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.463980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 936/1887] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:54.466239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.467587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.485759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de0eed-e03e-4f5f-9033-c6e76cd2c17c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.492295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.493018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.493307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.493845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] PATH_INFO: `/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.496209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b066d71fee1f4ab1b4382f9e8d2aa398)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.498881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.500378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa498a1b4cea418db8c76c6dd560ba11'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.508343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa498a1b4cea418db8c76c6dd560ba11'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.534034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa498a1b4cea418db8c76c6dd560ba11'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.535470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa498a1b4cea418db8c76c6dd560ba11'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.536108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa498a1b4cea418db8c76c6dd560ba11'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.563510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 952/1888] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:54 2022] DELETE /identity/v3/users/aa498a1b4cea418db8c76c6dd560ba11 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:54.565687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.567467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.570322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b066d71fee1f4ab1b4382f9e8d2aa398', 'request_id': 'req-594c824b-24ea-4c65-aee1-be5e5799fd17'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.576976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b066d71fee1f4ab1b4382f9e8d2aa398', 'request_id': 'req-594c824b-24ea-4c65-aee1-be5e5799fd17'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.585151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b066d71fee1f4ab1b4382f9e8d2aa398', 'request_id': 'req-594c824b-24ea-4c65-aee1-be5e5799fd17'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.602369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] The token cache is being invalidate because project b066d71fee1f4ab1b4382f9e8d2aa398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:54.603002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.604191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-594c824b-24ea-4c65-aee1-be5e5799fd17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:54.606923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 937/1889] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:54 2022] DELETE /identity/v3/projects/b066d71fee1f4ab1b4382f9e8d2aa398 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:54.614898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62920f-54c3-4481-8621-5e0b2b3086ba None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.615658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62920f-54c3-4481-8621-5e0b2b3086ba None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.616227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62920f-54c3-4481-8621-5e0b2b3086ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.642247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-eb62920f-54c3-4481-8621-5e0b2b3086ba None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:54.643494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eb62920f-54c3-4481-8621-5e0b2b3086ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.686040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1242ad6-41c2-4d80-9dea-6ef6a52a9761 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.692294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.692770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.693051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.693335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.693967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.695602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.701766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:54.720784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 953/1890] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:54.724779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.727390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.738683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 938/1891] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:54.741652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f38c126-0216-444f-b625-efa1b6fe6644 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.746221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.746550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.746956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.748283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.749016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.749310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.749590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] PATH_INFO: `/v3/projects/0b201b82515c40488aae1d5536dcd415/users/4e0a820c635c4a92852c395fb1a09818/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.766536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b201b82515c40488aae1d5536dcd415, user_id=4e0a820c635c4a92852c395fb1a09818, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.769377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.771351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:54.773145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:54.783573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 939/1892] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:54.788394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 954/1893] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:54 2022] PUT /identity/v3/projects/0b201b82515c40488aae1d5536dcd415/users/4e0a820c635c4a92852c395fb1a09818/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:54.789502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.795011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.805204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b0605b-1a1f-415f-bf61-46c5b11f53fd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.871185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:54.886862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.887588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.887936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.888221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.889149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:54.889520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.890367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab33567-ca9d-44cb-8675-c0cc411f1cd5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.892137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.895388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.895811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.896092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.896361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.897006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.898901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.906781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 940/1894] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:54.909611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 955/1895] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:54.911692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.915093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:54.923091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a63d491-5d5d-411b-8d51-7e99c5985ea5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.927920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.928246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4533d6-64c8-40d0-91d8-c12052d9a1b6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:54.928673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.929208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.929742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.930619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.932396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.934095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:54.935386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.935863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.936342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] PATH_INFO: `/v3/projects/0b201b82515c40488aae1d5536dcd415/users/4e0a820c635c4a92852c395fb1a09818/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.938472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b201b82515c40488aae1d5536dcd415, user_id=4e0a820c635c4a92852c395fb1a09818, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:54.941324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b878ac3d-7ed6-4933-b4a0-0e54483671bf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:54.954647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 956/1896] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:54 2022] PUT /identity/v3/projects/0b201b82515c40488aae1d5536dcd415/users/4e0a820c635c4a92852c395fb1a09818/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:54.965799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:54.965981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 941/1897] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:54.966389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:54.966961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:54.972678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.012806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] MFA Rules not processed for user `4e0a820c635c4a92852c395fb1a09818`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:55.014684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.080303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd64c1a-d4cf-4efb-bf1c-e8ff8b2578e8 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.085655 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.086351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.086728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.087007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.087626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.090129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.097598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 957/1898] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:55.098437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.099778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.114397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b466db-7778-4a5f-965c-1b6b48f4d60a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.121912 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 942/1899] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:54 2022] POST /identity/v3/users => generated 293 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:55.122059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.122155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.122256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.122256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.122256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] PATH_INFO: `/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.122256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae3f205ed7d4594a41f1f09b61f6da9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.122507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.123735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.166345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:55.177646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae3f205ed7d4594a41f1f09b61f6da9', 'request_id': 'req-d9c1680b-a06e-49e1-9342-518d3cad9291'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:55.184901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae3f205ed7d4594a41f1f09b61f6da9', 'request_id': 'req-d9c1680b-a06e-49e1-9342-518d3cad9291'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:55.191005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc64a7-5095-4e3d-8c06-ef6f6f745a5c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.194275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae3f205ed7d4594a41f1f09b61f6da9', 'request_id': 'req-d9c1680b-a06e-49e1-9342-518d3cad9291'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:55.206642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] The token cache is being invalidate because project 6ae3f205ed7d4594a41f1f09b61f6da9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:55.206923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:55.207582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:55.209090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 958/1900] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:55 2022] DELETE /identity/v3/projects/6ae3f205ed7d4594a41f1f09b61f6da9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:55.213248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.214652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.255471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.256241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.256527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.256933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.258772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.267458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:55.307137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1680b-a06e-49e1-9342-518d3cad9291 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.312772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.313376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.313654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.313925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.314589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.316402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc1c193-425a-4d0a-86a2-683b4f84f781 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.335470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 959/1901] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:55.336997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f3c6c-0b40-48b3-8771-935e79c1e140 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.337291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f3c6c-0b40-48b3-8771-935e79c1e140 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.337571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f3c6c-0b40-48b3-8771-935e79c1e140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.362980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.362980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.374657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 943/1902] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:55.376149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.376864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-454f3c6c-0b40-48b3-8771-935e79c1e140 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:55.378073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-454f3c6c-0b40-48b3-8771-935e79c1e140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.387696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 960/1903] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:55.397106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.397387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.397685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.401478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e61601-7ba0-4120-b3d2-319cdfd029f9 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.407352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.407979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.408268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.408621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.409771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.412639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.420966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:55.422919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.428694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 944/1904] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:55.437515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.445396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 961/1905] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:55.458396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.458484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93529bc-398d-4843-b489-675a917adff4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.460028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4f64d-f429-45bc-9718-2d4385fa9364 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.466752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.467327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.467833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.468111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.468321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.468419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.468869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.469016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:13:55.469469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] PATH_INFO: `/v3/projects/726af834191d4ade917d3007dda5f727/users/b97e68b3c2b744f9905dfc213d13a978/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.469572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.471795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.490777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 962/1906] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:13:55.494176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.518133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff4ea3-c0ec-47dd-9de4-ebacbd65a388 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.519473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726af834191d4ade917d3007dda5f727, user_id=b97e68b3c2b744f9905dfc213d13a978, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.522250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66693403-857b-4048-b20e-12f02dced580 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.523721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.524396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.525162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.525711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.526445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.528796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.552670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 945/1907] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:55 2022] PUT /identity/v3/projects/726af834191d4ade917d3007dda5f727/users/b97e68b3c2b744f9905dfc213d13a978/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:55.562142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.562435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.562754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.578324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 963/1908] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:13:55.582593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.597455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] MFA Rules not processed for user `b97e68b3c2b744f9905dfc213d13a978`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:55.601082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:55.727427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91d8c2-95ba-48b7-88f5-cbccaa73d59d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.743075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.743812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.743917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:55.744110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.746158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.749126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.752208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.770771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 946/1909] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:13:55.779897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.800018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 964/1910] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:55.804465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.807206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ef1e1-c90c-427f-a986-2f0a0b6322a4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.819214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823deb4-b560-42cd-95cf-2cb280338474 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.824105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.824728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.826675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.826975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.828964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.831292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.831919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.832205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.832569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.833232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.840146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.840478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.847742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 965/1911] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:55.858729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.877484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0bad4f-a659-4f4e-9697-5bf8dfdb54f2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.883782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:55.884513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:55.884893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:55.885267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:55.886782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:55.888875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:55.898958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 947/1912] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:55.902145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 966/1913] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:55.905077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.905608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:55.924221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160decc9-e24b-41ce-a36b-14b814159141 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:55.991612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:56.013062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.013910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.014349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.014741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] PATH_INFO: `/v3/projects/3ff2567834b24390a5d751400ef34039/users/5daf720a49f34497a4bb1d4f130d0486/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.048612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ff2567834b24390a5d751400ef34039, user_id=5daf720a49f34497a4bb1d4f130d0486, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.051687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd63d83-8b12-4010-a307-f129f5d32b5d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.057746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee91e2-a65e-4dcf-b534-47170dee62ed admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.063248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.064092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.064830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.064830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.065758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.068441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.082071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 967/1914] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:55 2022] PUT /identity/v3/projects/3ff2567834b24390a5d751400ef34039/users/5daf720a49f34497a4bb1d4f130d0486/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:56.097469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.098133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.098386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.112249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 948/1915] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:55 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:56.117228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.142267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b92adc-b331-4e07-9c46-12fe9a49c1a8 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.152205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] MFA Rules not processed for user `5daf720a49f34497a4bb1d4f130d0486`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:56.153463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:56.233137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.234064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.234424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.235066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.236069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.237919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.242282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 968/1916] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:56.243590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.251274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 949/1917] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:56.255227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.255712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be4bb7-b38f-4e26-8381-8812977d6f99 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.262684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.263291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.263709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.263982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.264727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.266660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.270700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b84689-7d46-4734-a9d6-eaba1905ec6d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.279639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.279639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.279639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.279639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.280109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.281979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.296758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 950/1918] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:56.300817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.320433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764ead2-7d24-4845-82a9-6e9c73474377 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.328206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 969/1919] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:56.329238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.334022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.334762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.335058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.335331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] PATH_INFO: `/v3/projects/5e457c3fd1aa46e38fa6690caa237186/users/6dbb8c6e891542f4aa5f1c340967df52/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.357015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e457c3fd1aa46e38fa6690caa237186, user_id=6dbb8c6e891542f4aa5f1c340967df52, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.360085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19954d82-65fa-442f-902a-34c59eb24d64 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.386868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 951/1920] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:56 2022] PUT /identity/v3/projects/5e457c3fd1aa46e38fa6690caa237186/users/6dbb8c6e891542f4aa5f1c340967df52/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:56.397561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.398799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.399171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.423375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c2dac-794b-4e94-aac6-7501c999f6a3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.437811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] MFA Rules not processed for user `6dbb8c6e891542f4aa5f1c340967df52`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:56.440072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:56.463013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:56.502819 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:56.537755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.538389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.538700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.538979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.539615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.541915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.559869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 952/1921] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:56.561284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.563397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:56.569289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 970/1922] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:56.569289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.580563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd6d72-663f-4ee0-96c8-7070b234cb93 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.586331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.587825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.588184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.588733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.590703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.593178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.631542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 971/1923] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:56.634983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.662955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3eba27-3abe-4f00-9e9b-0be35e20412e None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.727080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a31be6a-b291-4c4f-88a6-5e0150887008 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.735489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.736168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.736456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.736728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.737383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.739100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.744116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.744753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.745262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.745552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.747399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:56.750624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 972/1924] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:56.750624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.760675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f411aa75-5b79-4444-9d9b-57be7349c2f1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.765476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.766075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.766386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.766648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.767266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.769140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.791666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 973/1925] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:13:56 2022] POST /identity/v3/users => generated 298 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:13:56.793570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.803485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e667e-edbd-4144-9a8e-bcb4aa2aa5df admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.808729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.809382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.809665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.809940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] PATH_INFO: `/v3/projects/115492557b0647359db08752f49b6f1f/users/5dfb1396fe8849a48e4b1938a3bb0c0a/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.819315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.820888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.829051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115492557b0647359db08752f49b6f1f, user_id=5dfb1396fe8849a48e4b1938a3bb0c0a, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.830485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 953/1926] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:56.831881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.836660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.853905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f12340-922a-46ac-92c1-0338dcc3220a service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.854062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 974/1927] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:56 2022] PUT /identity/v3/projects/115492557b0647359db08752f49b6f1f/users/5dfb1396fe8849a48e4b1938a3bb0c0a/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:56.855179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.876363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:56.927473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.928502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.928871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.929151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.930225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.932241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.943896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25682-6fe2-45b7-b6a2-a97aa1bbbff4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.945866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 954/1928] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:56.950292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.950464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.951057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.951425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.951945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.952798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.954441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.963531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9176a46c-d7c1-458c-9ea3-c16c422b5de4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.969674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.970495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.970858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.971141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] PATH_INFO: `/v3/projects/f5598d782eac41c28b9d1fab12b295c4/users/ecc13b43f8a44c989f53420523ee0daf/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:56.974692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 975/1929] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:13:56.975786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:56.990897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5598d782eac41c28b9d1fab12b295c4, user_id=ecc13b43f8a44c989f53420523ee0daf, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:56.992039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683ea8-792e-41bc-b253-f4af03323929 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:56.993579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:56.997335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:56.998242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:56.998384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:56.998701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] PATH_INFO: `/v3/projects/115492557b0647359db08752f49b6f1f/users/5dfb1396fe8849a48e4b1938a3bb0c0a/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.001247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=115492557b0647359db08752f49b6f1f, user_id=5dfb1396fe8849a48e4b1938a3bb0c0a, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.005177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a42c9-b810-40b8-8bfd-53cc77c54a55 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.009135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:57.016032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 955/1930] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:56 2022] PUT /identity/v3/projects/f5598d782eac41c28b9d1fab12b295c4/users/ecc13b43f8a44c989f53420523ee0daf/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:57.020057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:57.023344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 976/1931] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:56 2022] PUT /identity/v3/projects/115492557b0647359db08752f49b6f1f/users/5dfb1396fe8849a48e4b1938a3bb0c0a/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:57.037227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.037527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.037809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.107079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] MFA Rules not processed for user `5dfb1396fe8849a48e4b1938a3bb0c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:57.110498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:57.136712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8fb97-fc02-4600-b129-506d8e86d19e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:57.213460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 977/1932] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:57.235421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:57.254769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:57.256380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.256380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.256380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.257049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.259029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.276155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 956/1933] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:13:57.277152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:57.299270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1d17-9f5a-4251-b5c3-0ca4e6611a2c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:57.306502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:57.307211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.307514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.308359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] PATH_INFO: `/v3/projects/f5598d782eac41c28b9d1fab12b295c4/users/ecc13b43f8a44c989f53420523ee0daf/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.311235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5598d782eac41c28b9d1fab12b295c4, user_id=ecc13b43f8a44c989f53420523ee0daf, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.314610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d732404a-4bd2-49a6-b977-28b6b6cfc469 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.327119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 957/1934] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:13:57 2022] PUT /identity/v3/projects/f5598d782eac41c28b9d1fab12b295c4/users/ecc13b43f8a44c989f53420523ee0daf/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:57.337641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.337920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.338257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.340152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7a2ba-0f04-4dcd-b2a2-962126b1cd5d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:57.382559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] MFA Rules not processed for user `ecc13b43f8a44c989f53420523ee0daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:57.383855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:57.399022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:57.417903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:57.418521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.418825 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.419088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.465213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 958/1935] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:57.471713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:57.477178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:57.478596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:57.488162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.489597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.490803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ba87cd-388d-48cf-9940-b00f9d81f122 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:57.494224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:57.495906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.496178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.496461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.500148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 978/1936] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:57.566190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.567709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.585792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 959/1937] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:57.596757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:57.608789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a946bad-0bd6-47c8-8811-5190bb427c3f service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:57.612930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:57.613545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:57.613824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:57.614093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:57.692975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:57.694314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9bf5a-bdf1-4860-a91a-7f62af04e84b service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:57.704768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 979/1938] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:13:58.081806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:58.083237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:58.147275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14cb4a-5a96-4867-9a37-9a79d3014c67 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:58.151847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:58.152467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:58.152747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:58.153029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] PATH_INFO: `/v3/users/64a1b90d4c004a55ac1a213060b2bb83` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:58.154131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a1b90d4c004a55ac1a213060b2bb83)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:58.156055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:58.201197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a1b90d4c004a55ac1a213060b2bb83'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:58.205553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:58.209518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a1b90d4c004a55ac1a213060b2bb83'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:58.216952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a1b90d4c004a55ac1a213060b2bb83'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:58.217693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a1b90d4c004a55ac1a213060b2bb83'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:58.218299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a1b90d4c004a55ac1a213060b2bb83'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:58.235488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 960/1939] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:58 2022] DELETE /identity/v3/users/64a1b90d4c004a55ac1a213060b2bb83 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:58.240116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:58.240593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:58.240700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:58.259607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:13:58.261385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:58.318780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 980/1940] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:13:58.329753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:58.331325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:58.397895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b991-7618-47ee-bcbd-7b144f503df2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:58.402008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:58.402725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:58.403079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:58.403431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:58.413463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:58.415059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:58.425207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 961/1941] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:58.855914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:58.858004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:58.880935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14196b50-2e6e-44ec-94a3-0c9fa92091fe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:58.886983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:58.887722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:58.888080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:58.888450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] PATH_INFO: `/v3/projects/604ff564e09641e9b4f3921e2fde2ab4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:58.889745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=604ff564e09641e9b4f3921e2fde2ab4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:58.892001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:59.007055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '604ff564e09641e9b4f3921e2fde2ab4', 'request_id': 'req-c925eb0a-0715-4166-b20b-91a707e4ace4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.032781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '604ff564e09641e9b4f3921e2fde2ab4', 'request_id': 'req-c925eb0a-0715-4166-b20b-91a707e4ace4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.061087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '604ff564e09641e9b4f3921e2fde2ab4', 'request_id': 'req-c925eb0a-0715-4166-b20b-91a707e4ace4'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.093778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] The token cache is being invalidate because project 604ff564e09641e9b4f3921e2fde2ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:13:59.094634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.096545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.106978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 981/1942] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:58 2022] DELETE /identity/v3/projects/604ff564e09641e9b4f3921e2fde2ab4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:13:59.107721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:59.121763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2a4130-c65d-40d1-8464-8f8da3ccda3c service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:59.243635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:59.246369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:59.246683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:59.246958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] PATH_INFO: `/v3/users/16b0bf813e2b46f182b59a864e054dee` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:59.251695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b0bf813e2b46f182b59a864e054dee)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:59.253822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:59.327304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b0bf813e2b46f182b59a864e054dee'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.345802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b0bf813e2b46f182b59a864e054dee'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.364213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b0bf813e2b46f182b59a864e054dee'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.368162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b0bf813e2b46f182b59a864e054dee'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.372074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b0bf813e2b46f182b59a864e054dee'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:13:59.394207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:59.418764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 962/1943] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:13:59 2022] DELETE /identity/v3/users/16b0bf813e2b46f182b59a864e054dee => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:13:59.505620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:59.507107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:59.518453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:13:59.579994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925eb0a-0715-4166-b20b-91a707e4ace4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:59.641505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:59.642240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:59.642673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:59.642975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:59.645092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:59.717829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:59.719765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803afb8-cc7f-4b18-bbcb-86075f96a181 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:13:59.730552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 982/1944] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:13:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:13:59.820307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:13:59.824017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:13:59.956139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead4630-c154-4837-ac60-58914ea10ccb admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:13:59.967405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:13:59.968701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:13:59.969134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:13:59.969519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] PATH_INFO: `/v3/projects/b224f8048c274d5187f1b7b00f99542d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:13:59.971437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b224f8048c274d5187f1b7b00f99542d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:13:59.973558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.061123 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b224f8048c274d5187f1b7b00f99542d', 'request_id': 'req-9dbabd3f-d2b9-450e-bf69-07172f9b010c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.069443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b224f8048c274d5187f1b7b00f99542d', 'request_id': 'req-9dbabd3f-d2b9-450e-bf69-07172f9b010c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.083436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b224f8048c274d5187f1b7b00f99542d', 'request_id': 'req-9dbabd3f-d2b9-450e-bf69-07172f9b010c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.113930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] The token cache is being invalidate because project b224f8048c274d5187f1b7b00f99542d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:00.127252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.127252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9dbabd3f-d2b9-450e-bf69-07172f9b010c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.127252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 963/1945] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:13:59 2022] DELETE /identity/v3/projects/b224f8048c274d5187f1b7b00f99542d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:00.127502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.127502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.127502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.148864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:00.150287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.246074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 983/1946] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:00.255241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.255723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.256014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.280332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:00.281583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.290204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 964/1947] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:00.295041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.296092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.297480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.306758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e81843-59ff-4f7b-88af-93a7641b52e2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.312861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.313479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.313873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.314154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.314363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8efeb-777f-4b4f-9e62-faf047b0d073 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.314847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:00.315280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.317005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.321801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.322886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.323540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.324009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] PATH_INFO: `/v3/users/5daf720a49f34497a4bb1d4f130d0486` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.325373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5daf720a49f34497a4bb1d4f130d0486)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.328115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.333884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 984/1948] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:00.337726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.348841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc722ed-eeda-41c9-865e-4e545518253b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.353547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.354154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.355061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.355403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.355974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.357868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.363394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:00.392772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 985/1949] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:00.395233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5daf720a49f34497a4bb1d4f130d0486'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.396719 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.403894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5daf720a49f34497a4bb1d4f130d0486'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.415424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5daf720a49f34497a4bb1d4f130d0486'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.416118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5daf720a49f34497a4bb1d4f130d0486'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.416868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d154f1dc-67e3-4eb4-b874-8e79e6c13cea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5daf720a49f34497a4bb1d4f130d0486'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:00.425219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:00.446695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 965/1950] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:00 2022] DELETE /identity/v3/users/5daf720a49f34497a4bb1d4f130d0486 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:00.451366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.451773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.452211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.476707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:00.478046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91476745-a9fe-4ae8-b2bf-f379bc8d4109 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.478615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.547533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.548759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.549046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.549320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.549952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.551880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.552221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 966/1951] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:00.561886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.563527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.579576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 986/1952] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:00.584114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.588034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.607124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7330d1-cc7a-404b-9da2-9bc9b770ffbc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.613001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.613621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.613895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.614170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.615288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.617251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.628393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 987/1953] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:14:00.632138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.644221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be2ad9-e32b-4278-af60-abc6ccbcc102 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.644347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5bc15-4647-43ea-93ad-89027ae36bc8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.649594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.650234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.650669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.650784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.650955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.652025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.652184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.652434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.652775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.653782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.663823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.665354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.665712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 988/1954] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:14:00.670293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.676856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 967/1955] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:14:00.680592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87a9a3-429c-474d-be74-2b6e1813f882 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.686160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.686890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.687177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.687451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] PATH_INFO: `/v3/projects/a5a4a087aa474a3da829492383570e15/users/8051a29d0fa74a6caa886b26eedc6772/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.704103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a4a087aa474a3da829492383570e15, user_id=8051a29d0fa74a6caa886b26eedc6772, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.706904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33556294-1175-491e-aa3b-b7359a4329d8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.725218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 989/1956] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:00 2022] PUT /identity/v3/projects/a5a4a087aa474a3da829492383570e15/users/8051a29d0fa74a6caa886b26eedc6772/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:00.738597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.738597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.738855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.738855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.738855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.770446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] MFA Rules not processed for user `8051a29d0fa74a6caa886b26eedc6772`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:00.771711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.798250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:00.807089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a62fe9-bff9-4584-b290-1f8b5fbba6e6 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.857434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 990/1957] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:00.862111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:00.877963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:00.885907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.886868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.887733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.888161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.890297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:00.952022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2849da4-e804-488f-834f-b57c6d247139 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:00.956801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:00.957681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:00.957890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:00.958611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:00.959015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.961154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.981360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:00.983162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:00.992133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 991/1958] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:00.996487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.007597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 968/1959] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:01.019970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.021501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.041534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100ddc19-df27-4fbf-b19b-c688e501f7ac service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.096655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7bc1e6-0510-4d85-baf6-29dd1278184e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.103435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.104283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.105463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.106075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.106908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.114429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.156129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.157933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.158285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.158704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] PATH_INFO: `/v3/projects/3ff2567834b24390a5d751400ef34039` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.163247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ff2567834b24390a5d751400ef34039)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.167563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.170227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 992/1960] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:00 2022] POST /identity/v3/users => generated 299 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:01.187202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.221412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015bba7a-4b2e-4ae8-975e-4c32dfd19043 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.246363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.253751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.255509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.256031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.258564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.260734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.277562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ff2567834b24390a5d751400ef34039', 'request_id': 'req-79fb106f-ba7f-4b19-866e-3992a58bf84e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.287913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 993/1961] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:01.291164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ff2567834b24390a5d751400ef34039', 'request_id': 'req-79fb106f-ba7f-4b19-866e-3992a58bf84e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.304140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.321099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273367c-164b-494e-93dc-462b347db534 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.322965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ff2567834b24390a5d751400ef34039', 'request_id': 'req-79fb106f-ba7f-4b19-866e-3992a58bf84e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.346059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] The token cache is being invalidate because project 3ff2567834b24390a5d751400ef34039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:01.346832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.349019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.354200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 969/1962] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:01 2022] DELETE /identity/v3/projects/3ff2567834b24390a5d751400ef34039 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:01.358682 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.361973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.384916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:01.430461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:01.464317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.464317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.464636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.464857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] PATH_INFO: `/v3/projects/80e2a1baf06244209dffdde2a5550538/users/ed430b3cfa5746268c2882cc833a8f7b/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.499783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e2a1baf06244209dffdde2a5550538, user_id=ed430b3cfa5746268c2882cc833a8f7b, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.503408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb106f-ba7f-4b19-866e-3992a58bf84e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.504590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.518809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.518809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.518809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.518809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] PATH_INFO: `/v3/users/5dfb1396fe8849a48e4b1938a3bb0c0a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.519141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfb1396fe8849a48e4b1938a3bb0c0a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.521128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.537493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 994/1963] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:01 2022] PUT /identity/v3/projects/80e2a1baf06244209dffdde2a5550538/users/ed430b3cfa5746268c2882cc833a8f7b/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:01.541140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.543141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.625027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfb1396fe8849a48e4b1938a3bb0c0a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.634407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfb1396fe8849a48e4b1938a3bb0c0a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.654340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429fd3c6-c927-4d55-a584-2d6ab0d81de4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.655968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfb1396fe8849a48e4b1938a3bb0c0a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.657535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfb1396fe8849a48e4b1938a3bb0c0a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.658478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfb1396fe8849a48e4b1938a3bb0c0a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:01.659360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.659987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.660266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.660535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.661326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.663092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.673865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 995/1964] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:14:01.674762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.676705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.683448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 970/1965] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:01 2022] DELETE /identity/v3/users/5dfb1396fe8849a48e4b1938a3bb0c0a => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:01.687823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:01.689500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.769809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9e77b-b861-46a7-ad22-b44ac82715ed admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.873436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765fbc30-bae5-473b-acda-6669e4273e06 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:01.886103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.888297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.888581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.888908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] PATH_INFO: `/v3/projects/80e2a1baf06244209dffdde2a5550538/users/ed430b3cfa5746268c2882cc833a8f7b/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.895083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e2a1baf06244209dffdde2a5550538, user_id=ed430b3cfa5746268c2882cc833a8f7b, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:01.900851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dd71e6-2a97-4158-8a1e-5bdd7d41748b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:01.919511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:01.920269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.920564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.920843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.924299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:01.930178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 971/1966] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:01 2022] PUT /identity/v3/projects/80e2a1baf06244209dffdde2a5550538/users/ed430b3cfa5746268c2882cc833a8f7b/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:01.940974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399a33-e76d-4891-88b1-c644996797ea None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:01.941285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399a33-e76d-4891-88b1-c644996797ea None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:01.941751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47399a33-e76d-4891-88b1-c644996797ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:01.967209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:01.980447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-47399a33-e76d-4891-88b1-c644996797ea None None] MFA Rules not processed for user `ed430b3cfa5746268c2882cc833a8f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:01.981716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-47399a33-e76d-4891-88b1-c644996797ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.035589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:02.036337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.038228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.052020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 996/1967] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:02.119780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 972/1968] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:02.127802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.129295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.135157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399a33-e76d-4891-88b1-c644996797ea None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.136619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-47399a33-e76d-4891-88b1-c644996797ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.145877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbac5a1-0140-4a21-9fc6-7535b8f9b333 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.203327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.204158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.204449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.204746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.206509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.218486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47399a33-e76d-4891-88b1-c644996797ea None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.224317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.224965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.225253 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.225538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] PATH_INFO: `/v3/projects/115492557b0647359db08752f49b6f1f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.226779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=115492557b0647359db08752f49b6f1f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.228829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.302998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.304521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cba61f-0798-4477-9bd4-d5e2a238760f service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.316964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '115492557b0647359db08752f49b6f1f', 'request_id': 'req-a99601ef-b245-43d2-bdf8-5aebecfd3692'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:02.318940 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 997/1969] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:02 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:02.323848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '115492557b0647359db08752f49b6f1f', 'request_id': 'req-a99601ef-b245-43d2-bdf8-5aebecfd3692'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:02.333793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '115492557b0647359db08752f49b6f1f', 'request_id': 'req-a99601ef-b245-43d2-bdf8-5aebecfd3692'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:02.351536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] The token cache is being invalidate because project 115492557b0647359db08752f49b6f1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:02.351853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:02.354284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a99601ef-b245-43d2-bdf8-5aebecfd3692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:02.361677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.361836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.362051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.365128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 973/1970] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:02 2022] DELETE /identity/v3/projects/115492557b0647359db08752f49b6f1f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:02.407821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:02.409656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.533373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 998/1971] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:02.541075 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.541565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.541707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.561706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:02.563047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:02.572291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 974/1972] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:02.577227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.588355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd00056-7e9f-42c0-8c37-b45096895cc2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.594278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.595064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.595332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.595667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.596523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:02.596954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.598798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.608952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:02.614743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 999/1973] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:14:02.617441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.628532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d97ef-a36b-4e88-bca8-609c201ac0cf None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.635004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.636616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.637177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.637324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.637967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.640302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.671486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 975/1974] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:02 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:02.676415 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.689127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc7121-b08a-4aa6-a844-3177e4520fa2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.760361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.761549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.770670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.770670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.770670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.770670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.795499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1000/1975] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:02 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:02.799720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.813633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99caf27d-b526-4b61-811e-6a800093ca3d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.819906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.820756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.821137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.821536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.822282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.825792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.844348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 976/1976] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:14:02.845955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.876032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e68a8-fe74-4be5-b393-0e3765189f08 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.886837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.888803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.889109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.889393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.892238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.894239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe7432-015f-4529-b6b5-83c73b872b27 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:02.914513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1001/1977] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:02.917764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:02.931488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff0105-188b-4824-a45e-46b87ff66194 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:02.941287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:02.942757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:02.943291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:02.943670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] PATH_INFO: `/v3/projects/a6f92c41bce9472fa5c20b8b9bbae3c8/users/342580a153bb4752a70abf15d0c1adf6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:02.971168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:02.987569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f92c41bce9472fa5c20b8b9bbae3c8, user_id=342580a153bb4752a70abf15d0c1adf6, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:02.999886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.034712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 977/1978] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:02 2022] PUT /identity/v3/projects/a6f92c41bce9472fa5c20b8b9bbae3c8/users/342580a153bb4752a70abf15d0c1adf6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:03.038111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.038463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.038839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.115742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] MFA Rules not processed for user `342580a153bb4752a70abf15d0c1adf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:03.118047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:03.270942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1002/1979] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:03.277201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.403599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f60474-1201-4e8a-a571-e8482f1b2b5d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.425189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.425189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.425189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.425189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.425549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.427015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.468876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 978/1980] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:03 2022] POST /identity/v3/projects => generated 369 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:03.473417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.487914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38cc7c-46d7-490e-9fa2-f05f4edc84fc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.584443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.585063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.585349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.585628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.586271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.588383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.606784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:03.614602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1003/1981] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:03 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:03.617620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.628766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d9fae2-9278-4e03-9425-54a6a4f8d2aa admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.635062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.635711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.636124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.639129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.640272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.641897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.653903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 979/1982] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:03.658368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.669697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c371819-9fe0-46a9-a98f-f957364de090 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.674823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.675665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.676008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.676458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] PATH_INFO: `/v3/projects/ef19cd545cce4063815f15f81326278a/users/49da22dba65841a98be7f7d5f55d7d5f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.694222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef19cd545cce4063815f15f81326278a, user_id=49da22dba65841a98be7f7d5f55d7d5f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.696890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.710356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1004/1983] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:03 2022] PUT /identity/v3/projects/ef19cd545cce4063815f15f81326278a/users/49da22dba65841a98be7f7d5f55d7d5f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:03.714853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.724758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f221-81e5-4330-923e-21527410f26b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.781114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.781864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.782147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.782424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.783111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.784716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b006d2-c4ff-45b4-9965-c43533bd1605 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.796993 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 980/1984] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:03.799471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.811316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb95da2-680d-4c89-b3f1-9a918134017b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:03.816121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:03.816884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.817271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.817655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] PATH_INFO: `/v3/projects/ef19cd545cce4063815f15f81326278a/users/49da22dba65841a98be7f7d5f55d7d5f/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.819772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef19cd545cce4063815f15f81326278a, user_id=49da22dba65841a98be7f7d5f55d7d5f, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:03.822490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:03.833219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1005/1985] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:03 2022] PUT /identity/v3/projects/ef19cd545cce4063815f15f81326278a/users/49da22dba65841a98be7f7d5f55d7d5f/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:03.844390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:03.844685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:03.845066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:03.875402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] MFA Rules not processed for user `49da22dba65841a98be7f7d5f55d7d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:03.876608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:03.923338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:03.944595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 981/1986] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:03.955836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:03.957539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:04.014157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab3ad7-216c-4f7c-ac0d-c4f89668b7c2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:04.017741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:04.018326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:04.018637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:04.018912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:04.028121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:04.029520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:04.038913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1006/1987] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:04.082400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:04.083840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:04.096466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9795e7-0d60-4587-a530-e248c1e9aed4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:04.100064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:04.100657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:04.100939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:04.101217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:04.158345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:04.159827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:04.169808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 982/1988] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:06.640902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:06.642789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:06.805538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e8db4-3ebc-4f14-9f84-d16b10a7aee5 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:06.811687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:06.812442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:06.812817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:06.813188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] PATH_INFO: `/v3/users/920e1914cddd453d9654246158367c3c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:06.815030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=920e1914cddd453d9654246158367c3c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:06.817033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:06.924042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920e1914cddd453d9654246158367c3c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:06.946056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:06.947245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920e1914cddd453d9654246158367c3c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:06.947501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:06.958506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920e1914cddd453d9654246158367c3c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:06.959564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920e1914cddd453d9654246158367c3c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:06.960993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-495d48a9-bfe8-43ee-9d02-3f05d501b59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920e1914cddd453d9654246158367c3c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:06.963383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:06.973761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477d1e0-9e6b-4939-b640-a064fbf1085b service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:06.991016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1007/1989] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:06 2022] DELETE /identity/v3/users/920e1914cddd453d9654246158367c3c => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:06.992504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:06.993355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:06.993891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:06.994493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] PATH_INFO: `/v3/users/b97e68b3c2b744f9905dfc213d13a978` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:06.996017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:06.996376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:06.996727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:06.997022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97e68b3c2b744f9905dfc213d13a978)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:06.999435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:07.020711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:07.021979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:07.063015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97e68b3c2b744f9905dfc213d13a978'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:07.071971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97e68b3c2b744f9905dfc213d13a978'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:07.085098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97e68b3c2b744f9905dfc213d13a978'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:07.086351 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97e68b3c2b744f9905dfc213d13a978'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:07.087417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9d642da7-d49b-4bcf-abe3-008957b4ae82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97e68b3c2b744f9905dfc213d13a978'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:07.105301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1008/1990] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:07.114703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:07.116212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:07.119795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 983/1991] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:06 2022] DELETE /identity/v3/users/b97e68b3c2b744f9905dfc213d13a978 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:07.124697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:07.125637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:07.126064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:07.150235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:07.152649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:07.189874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b330e-794b-4519-ba97-3b51caa219d9 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:07.302806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:07.303648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:07.304286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:07.305345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:07.317122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:07.347390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 984/1992] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:07.354031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:07.367940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958574bb-7554-45bd-bdf8-353cf56d7718 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:07.380414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:07.381273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:07.382279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:07.382732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:07.395110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:07.399968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:07.403108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:07.420833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:07.425906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 985/1993] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:07.438064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:07.464914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1009/1994] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:14:07.714412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:07.722212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:07.825298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e6e00-35b1-4930-b596-ebfa937d08e4 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:07.838120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:07.839305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:07.839589 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:07.839862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:07.841703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:07.959997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:07.959997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:07.970595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 986/1995] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:08.487977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:08.489659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:08.513755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36a108-bd57-4471-86df-b6aeb2d85bc0 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:08.538630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:08.538630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:08.538630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:08.538630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] PATH_INFO: `/v3/projects/9f3781c122ed429aaaf93f42d43180e0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:08.539057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3781c122ed429aaaf93f42d43180e0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:08.539057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:08.658216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3781c122ed429aaaf93f42d43180e0', 'request_id': 'req-3d25282c-c326-4e86-aff7-675a7bb88a66'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:08.684782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3781c122ed429aaaf93f42d43180e0', 'request_id': 'req-3d25282c-c326-4e86-aff7-675a7bb88a66'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:08.704515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3781c122ed429aaaf93f42d43180e0', 'request_id': 'req-3d25282c-c326-4e86-aff7-675a7bb88a66'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:08.733847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] The token cache is being invalidate because project 9f3781c122ed429aaaf93f42d43180e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:08.733847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:08.743110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:08.748978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1010/1996] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:08 2022] DELETE /identity/v3/projects/9f3781c122ed429aaaf93f42d43180e0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:08.749797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:08.766450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb249c-7195-401e-ab47-6a7c89c03f11 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:08.855576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:08.856920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:08.857538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:08.858165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] PATH_INFO: `/v3/users/f3a032e7b06042688389c1cb4da511db` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:08.862190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a032e7b06042688389c1cb4da511db)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:08.872612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:08.958821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:08.985882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a032e7b06042688389c1cb4da511db'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.007087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a032e7b06042688389c1cb4da511db'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.028160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a032e7b06042688389c1cb4da511db'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.028935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a032e7b06042688389c1cb4da511db'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.029677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a032e7b06042688389c1cb4da511db'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.058852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 987/1997] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:08 2022] DELETE /identity/v3/users/f3a032e7b06042688389c1cb4da511db => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:09.101781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.108010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.139263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25282c-c326-4e86-aff7-675a7bb88a66 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:09.195668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.198906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.282116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:09.282807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:09.283091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:09.283364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] PATH_INFO: `/v3/projects/726af834191d4ade917d3007dda5f727` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:09.285209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=726af834191d4ade917d3007dda5f727)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:09.292994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:09.335947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffec4d9-ecc7-48a3-9e39-08dbdd73e849 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:09.348783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:09.349512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:09.351117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:09.351410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] PATH_INFO: `/v3/users/342580a153bb4752a70abf15d0c1adf6` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:09.355021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=342580a153bb4752a70abf15d0c1adf6)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:09.362240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:09.376144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726af834191d4ade917d3007dda5f727', 'request_id': 'req-218388bb-a2e3-4664-9f1c-9469f7e32bba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.385382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726af834191d4ade917d3007dda5f727', 'request_id': 'req-218388bb-a2e3-4664-9f1c-9469f7e32bba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.398490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726af834191d4ade917d3007dda5f727', 'request_id': 'req-218388bb-a2e3-4664-9f1c-9469f7e32bba'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.445134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] The token cache is being invalidate because project 726af834191d4ade917d3007dda5f727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:09.445480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.450309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.455861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '342580a153bb4752a70abf15d0c1adf6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.456998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1011/1998] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:09 2022] DELETE /identity/v3/projects/726af834191d4ade917d3007dda5f727 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:09.457866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.462012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.464593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '342580a153bb4752a70abf15d0c1adf6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.470812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:09.481203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '342580a153bb4752a70abf15d0c1adf6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.482199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '342580a153bb4752a70abf15d0c1adf6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.482994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1d212e3-10ca-44cd-911e-ad777f1459fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '342580a153bb4752a70abf15d0c1adf6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.522760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 988/1999] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:09 2022] DELETE /identity/v3/users/342580a153bb4752a70abf15d0c1adf6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:09.527857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:09.529428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:09.529428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:09.574345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:09.577788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.634309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218388bb-a2e3-4664-9f1c-9469f7e32bba admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:09.644336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:09.726140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 989/2000] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:09.737547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.742455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.773053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b7de6-3930-44cd-be58-5868ee9a9170 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:09.778951 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:09.779851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:09.780267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:09.780601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] PATH_INFO: `/v3/projects/cad42fb175464edea1f2145e2ff4b9e0` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:09.781922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad42fb175464edea1f2145e2ff4b9e0)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:09.783463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:09.784080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:09.784254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:09.784610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:09.786106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] PATH_INFO: `/v3/users/6dbb8c6e891542f4aa5f1c340967df52` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:09.787803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dbb8c6e891542f4aa5f1c340967df52)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:09.795299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:09.874756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dbb8c6e891542f4aa5f1c340967df52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.881490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad42fb175464edea1f2145e2ff4b9e0', 'request_id': 'req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.884922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dbb8c6e891542f4aa5f1c340967df52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.892621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad42fb175464edea1f2145e2ff4b9e0', 'request_id': 'req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.900432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dbb8c6e891542f4aa5f1c340967df52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.901541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dbb8c6e891542f4aa5f1c340967df52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.903229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dbb8c6e891542f4aa5f1c340967df52'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.908463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad42fb175464edea1f2145e2ff4b9e0', 'request_id': 'req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.927337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] The token cache is being invalidate because project cad42fb175464edea1f2145e2ff4b9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:09.927689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.928959 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:09.936726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 990/2001] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:09 2022] DELETE /identity/v3/projects/cad42fb175464edea1f2145e2ff4b9e0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:09.936726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.936961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1012/2002] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:09 2022] DELETE /identity/v3/users/6dbb8c6e891542f4aa5f1c340967df52 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:09.936961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:09.939991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:09.953198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.033699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614a43-13c1-4dc3-a9f7-87b7ffe5b1a5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:10.036102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:10.098739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f88ef-436e-4a14-90dd-83d96d52bdac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:10.107254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:10.108055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:10.108467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:10.108841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] PATH_INFO: `/v3/users/4e0a820c635c4a92852c395fb1a09818` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:10.110876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0a820c635c4a92852c395fb1a09818)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:10.113083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:10.128994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:10.129620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:10.129950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:10.130225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:10.132436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.147894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:10.149734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:10.165513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 991/2003] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:10.184802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0a820c635c4a92852c395fb1a09818'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.202384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0a820c635c4a92852c395fb1a09818'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.214708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0a820c635c4a92852c395fb1a09818'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.215532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0a820c635c4a92852c395fb1a09818'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.216219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0a820c635c4a92852c395fb1a09818'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.250625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1013/2004] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:09 2022] DELETE /identity/v3/users/4e0a820c635c4a92852c395fb1a09818 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:10.552541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:10.554354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.571078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe632f-c4e1-478c-a667-fd968ea596fe service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:10.585572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:10.658984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:10.659909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:10.660298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:10.660680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] PATH_INFO: `/v3/projects/5e457c3fd1aa46e38fa6690caa237186` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:10.661934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e457c3fd1aa46e38fa6690caa237186)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:10.664172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:10.724807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:10.728186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e457c3fd1aa46e38fa6690caa237186', 'request_id': 'req-42a95db9-0a91-4caa-a0ec-6d268fb92198'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.728506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.736656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e457c3fd1aa46e38fa6690caa237186', 'request_id': 'req-42a95db9-0a91-4caa-a0ec-6d268fb92198'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.746886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e457c3fd1aa46e38fa6690caa237186', 'request_id': 'req-42a95db9-0a91-4caa-a0ec-6d268fb92198'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.756829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f6544-216f-4bfc-ae2b-7f4b6a1537ea admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:10.767055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] The token cache is being invalidate because project 5e457c3fd1aa46e38fa6690caa237186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:10.767814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.769333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.771455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 992/2005] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:10 2022] DELETE /identity/v3/projects/5e457c3fd1aa46e38fa6690caa237186 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:10.772318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:10.774477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.823464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:10.824102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:10.824385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:10.824661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] PATH_INFO: `/v3/projects/a6f92c41bce9472fa5c20b8b9bbae3c8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:10.825878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f92c41bce9472fa5c20b8b9bbae3c8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:10.827807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:10.843308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a95db9-0a91-4caa-a0ec-6d268fb92198 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:10.848241 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:10.848874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:10.849162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:10.849443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] PATH_INFO: `/v3/projects/0b201b82515c40488aae1d5536dcd415` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:10.850598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b201b82515c40488aae1d5536dcd415)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:10.852555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:10.884665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f92c41bce9472fa5c20b8b9bbae3c8', 'request_id': 'req-369af9a2-ad6a-46d9-80c1-fccff2585b3b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.885844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:10.891751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f92c41bce9472fa5c20b8b9bbae3c8', 'request_id': 'req-369af9a2-ad6a-46d9-80c1-fccff2585b3b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.899790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f92c41bce9472fa5c20b8b9bbae3c8', 'request_id': 'req-369af9a2-ad6a-46d9-80c1-fccff2585b3b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.900336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:10.902302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b201b82515c40488aae1d5536dcd415', 'request_id': 'req-73b44193-9f70-4737-a18b-799c96aff837'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.909603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b201b82515c40488aae1d5536dcd415', 'request_id': 'req-73b44193-9f70-4737-a18b-799c96aff837'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.913403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] The token cache is being invalidate because project a6f92c41bce9472fa5c20b8b9bbae3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:10.913697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.914394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.916302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1014/2006] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:10 2022] DELETE /identity/v3/projects/a6f92c41bce9472fa5c20b8b9bbae3c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:10.917318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:10.918031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b201b82515c40488aae1d5536dcd415', 'request_id': 'req-73b44193-9f70-4737-a18b-799c96aff837'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.920223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.930459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] The token cache is being invalidate because project 0b201b82515c40488aae1d5536dcd415 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:10.930920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.931567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:10.933585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 993/2007] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:10 2022] DELETE /identity/v3/projects/0b201b82515c40488aae1d5536dcd415 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:10.934345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:10.938927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:10.985934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369af9a2-ad6a-46d9-80c1-fccff2585b3b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.003254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b44193-9f70-4737-a18b-799c96aff837 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.008442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.009176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.009367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.009751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] PATH_INFO: `/v3/users/49da22dba65841a98be7f7d5f55d7d5f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.011867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49da22dba65841a98be7f7d5f55d7d5f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.013846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.059772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.061279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.061740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.062082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] PATH_INFO: `/v3/users/ecc13b43f8a44c989f53420523ee0daf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.063662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc13b43f8a44c989f53420523ee0daf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.065901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.079643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49da22dba65841a98be7f7d5f55d7d5f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.092614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49da22dba65841a98be7f7d5f55d7d5f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.102456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49da22dba65841a98be7f7d5f55d7d5f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.103150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49da22dba65841a98be7f7d5f55d7d5f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.103810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-4353ef49-e618-4396-92ce-839585bbbdf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49da22dba65841a98be7f7d5f55d7d5f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.119469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc13b43f8a44c989f53420523ee0daf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.125881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 994/2008] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:10 2022] DELETE /identity/v3/users/49da22dba65841a98be7f7d5f55d7d5f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:11.127184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.127546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.127921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.129427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc13b43f8a44c989f53420523ee0daf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.147078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc13b43f8a44c989f53420523ee0daf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.147965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc13b43f8a44c989f53420523ee0daf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.148725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b537be99-7cfb-42be-95a3-68cd53deab03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc13b43f8a44c989f53420523ee0daf'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.161120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.163158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.170342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1015/2009] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:10 2022] DELETE /identity/v3/users/ecc13b43f8a44c989f53420523ee0daf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:11.219644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:11.240988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 995/2010] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:11.242749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a592a-525e-4165-ad20-ed505289e826 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.243069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a592a-525e-4165-ad20-ed505289e826 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.243347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a592a-525e-4165-ad20-ed505289e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.253735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:11.266286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-609a592a-525e-4165-ad20-ed505289e826 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.267964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-609a592a-525e-4165-ad20-ed505289e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.357502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1016/2011] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:11.362382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.372470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456cfe-0c7d-471c-b7ce-6f0f191d5549 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.378179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.379112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.379265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.379646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.380308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:11.381247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.382840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.397811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 996/2012] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:11.403222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a592a-525e-4165-ad20-ed505289e826 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.413158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a592a-525e-4165-ad20-ed505289e826 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.419651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.420618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.420950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.421226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.421942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.427935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.465797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1017/2013] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:11.469531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.483119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9dad8-49b0-439e-bf5d-27a2d8a46c5e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.528990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.530960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.586317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.587238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.587514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.587785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.588581 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.594522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.609232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0a284-c811-4084-8719-ab9a941fcce7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.615102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.615881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.616158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.616431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] PATH_INFO: `/v3/projects/ef19cd545cce4063815f15f81326278a` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.617575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef19cd545cce4063815f15f81326278a)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.620785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.620901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 997/2014] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:11.621871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.623448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.638669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5642516-016e-4da0-ae12-41e9963bb5b2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.644699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.645538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.645856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.646186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] PATH_INFO: `/v3/projects/f5598d782eac41c28b9d1fab12b295c4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.647374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5598d782eac41c28b9d1fab12b295c4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.649274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.676687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef19cd545cce4063815f15f81326278a', 'request_id': 'req-66988e54-b92d-4ba9-9850-f2050af88e22'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.683912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef19cd545cce4063815f15f81326278a', 'request_id': 'req-66988e54-b92d-4ba9-9850-f2050af88e22'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.694680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef19cd545cce4063815f15f81326278a', 'request_id': 'req-66988e54-b92d-4ba9-9850-f2050af88e22'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.707743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] The token cache is being invalidate because project ef19cd545cce4063815f15f81326278a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:11.708061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.708961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.709853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5598d782eac41c28b9d1fab12b295c4', 'request_id': 'req-14c48ac3-a516-4fec-80d2-51ab18a6a45e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.710473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1018/2015] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:11 2022] DELETE /identity/v3/projects/ef19cd545cce4063815f15f81326278a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:11.711266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.712563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.715936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5598d782eac41c28b9d1fab12b295c4', 'request_id': 'req-14c48ac3-a516-4fec-80d2-51ab18a6a45e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.724032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5598d782eac41c28b9d1fab12b295c4', 'request_id': 'req-14c48ac3-a516-4fec-80d2-51ab18a6a45e'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.727377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:11.736189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] The token cache is being invalidate because project f5598d782eac41c28b9d1fab12b295c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:11.736464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.737553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14c48ac3-a516-4fec-80d2-51ab18a6a45e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:11.740211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 998/2016] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:11 2022] DELETE /identity/v3/projects/f5598d782eac41c28b9d1fab12b295c4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:11.742005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a7eb-5218-4ae8-a91d-a8a8a155836b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.742430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a7eb-5218-4ae8-a91d-a8a8a155836b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.742869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae22a7eb-5218-4ae8-a91d-a8a8a155836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.765920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ae22a7eb-5218-4ae8-a91d-a8a8a155836b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.767197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ae22a7eb-5218-4ae8-a91d-a8a8a155836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.797980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66988e54-b92d-4ba9-9850-f2050af88e22 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.822401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:11.838841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 999/2017] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:11.840427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3950c-d3e9-4965-af8d-e92b01b18285 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.840722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3950c-d3e9-4965-af8d-e92b01b18285 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.841001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3950c-d3e9-4965-af8d-e92b01b18285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.871971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-76b3950c-d3e9-4965-af8d-e92b01b18285 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.873414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-76b3950c-d3e9-4965-af8d-e92b01b18285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.876684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.877378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.877730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.878075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.878833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.880512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f2107-2827-43ba-8261-a544125ce8bb admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.887091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1000/2018] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:11.887091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.887091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.887091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.897257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1019/2019] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:11.899095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.899518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.899961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.906182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.907478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.921149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1001/2020] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:11.921149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.922491 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.923774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:11.925027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:11.937247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1020/2021] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:11.937247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.937504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b5539-e399-4392-b9a4-b79255bbe874 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.940019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.940646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.940988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.941265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.941914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.944554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.947499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86069e8-78bd-401a-b3ee-2586e3b57d6c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.952654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.953280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.953568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.953839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.954496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:11.954830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.958919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1002/2022] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:11.959106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.959189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.971823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f424e86-e6f8-410f-8116-2c2b665fac56 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.972536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1021/2023] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:11.977033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.977242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:11.977966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.978255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.978529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:11.979314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:11.979637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:11.981322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:11.991124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb545e-e10d-47b4-b847-3016c0b7088f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:11.995606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1003/2024] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:11.997244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:11.998020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:11.998374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:11.998765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] PATH_INFO: `/v3/projects/2b6a883384f640dcb6a8db0cc8d28635/users/2f9a114987a445f9888168a01e2e6e30/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.000188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.011256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318e6bd-497b-475c-8905-66cfc708f78e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.016442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.017072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.017364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.017500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6a883384f640dcb6a8db0cc8d28635, user_id=2f9a114987a445f9888168a01e2e6e30, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.017637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.018269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.020218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f27991-7bbb-41d6-929e-326e78f7d9a9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.020483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.036465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1022/2025] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:11 2022] PUT /identity/v3/projects/2b6a883384f640dcb6a8db0cc8d28635/users/2f9a114987a445f9888168a01e2e6e30/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:12.037175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.074059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1004/2026] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:11 2022] POST /identity/v3/projects => generated 401 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.077410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.077897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.078652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.114925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] MFA Rules not processed for user `2f9a114987a445f9888168a01e2e6e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:12.116896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:12.118901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59850e06-d8d0-4dfc-8e7f-dbf7d00a6d11 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.141642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:12.190142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.190917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.191197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.191469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.192093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.193970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.204311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1005/2027] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:12.205205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.216431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84fe608-30f6-4924-bb2d-59a9b0e55ca8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.221650 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.222166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1023/2028] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.222558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.222885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.223047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.223143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.223752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.225669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.261733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1006/2029] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/users => generated 301 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.262757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.273671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61233a-2199-4d74-831c-9e26b594f385 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.302488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847859e-3791-4b5c-bf69-3cec17bbae36 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.308869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.309492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.309832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.310283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.311091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.313062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.343551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.344453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.344736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.345011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.345663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.347521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1024/2030] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/projects => generated 393 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.347824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.348428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.364561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:12.374614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1007/2031] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.375014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.384867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb707c-eab7-4bc9-a86f-efdccea954ba admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.430648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea446346-7bc7-4640-80f3-cc9a4698f31e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.435588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.436267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.436621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.436903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.437558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.439192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.442054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.442720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.443006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.443280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.443903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.445848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.449472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1025/2032] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.450226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.459849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991083f-db35-4036-9fe6-3478d09e0d0f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.465418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.466011 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.466294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.466589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.467244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.467364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1008/2033] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.468281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.468910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.478455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c895f-7dc3-4d43-aac3-f3cabcebad7f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.483420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1026/2034] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.483420 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.486621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.487230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.487512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.487806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.488475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.490102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.491396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b151280-5ce2-493b-ae57-caf96db4f982 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.495965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.496537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.496814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.497086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.497712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.499304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.500344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1009/2035] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.501149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.511111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6f961-ca6e-4347-85f0-5af27d259a8a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.514564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1027/2036] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.517802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.519565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.520163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.520458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.520703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.521335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.522246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:12.522936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.526938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998b8a6d-fb36-4f92-a578-e7f6a6ae986f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.532691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.533999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.534289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.534566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] PATH_INFO: `/v3/projects/1fa237924dec409fa915a68666dc4010/users/6f32f988241449bfaaee1a5406676391/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.535079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1010/2037] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.535906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.548990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6fdc51-ef18-494b-970f-1038ead9c12e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.550488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fa237924dec409fa915a68666dc4010, user_id=6f32f988241449bfaaee1a5406676391, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.553217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.553677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.554318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.554626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.555155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] PATH_INFO: `/v3/projects/d07a18118f704d6a83a13262c918df21/users/4fbbb2241f3941a6aa199ddc6ebc479c/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.566364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1028/2038] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:12 2022] PUT /identity/v3/projects/1fa237924dec409fa915a68666dc4010/users/6f32f988241449bfaaee1a5406676391/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:12.567242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.580092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07a18118f704d6a83a13262c918df21, user_id=4fbbb2241f3941a6aa199ddc6ebc479c, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.582851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007aaeea-2336-4d7e-83d6-4aaae63edab0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.598239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1011/2039] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:12 2022] PUT /identity/v3/projects/d07a18118f704d6a83a13262c918df21/users/4fbbb2241f3941a6aa199ddc6ebc479c/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:12.599543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.599819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.600084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.628770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] MFA Rules not processed for user `6f32f988241449bfaaee1a5406676391`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:12.629970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:12.636766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff47513-4da9-4d37-ac86-fba2d9d1409f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.697005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.697765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.698759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.699099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] PATH_INFO: `/v3/projects/1b0c83c4cd1841db9e83459c754e3417/users/3ba9eda7f6fe4ebbbc5a176eadd5ea3e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.699311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1012/2040] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:12.703304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.714409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b0c83c4cd1841db9e83459c754e3417, user_id=3ba9eda7f6fe4ebbbc5a176eadd5ea3e, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.717093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.717450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0e034-16de-40aa-a385-def969ebab97 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.722482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.723266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.723555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.723831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.724588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.726476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af901d6-01f5-443c-a098-b07338050ce1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.730772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1029/2041] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:12 2022] PUT /identity/v3/projects/1b0c83c4cd1841db9e83459c754e3417/users/3ba9eda7f6fe4ebbbc5a176eadd5ea3e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:12.731347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.746990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1013/2042] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:12.746990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.746990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.746990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.773294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] MFA Rules not processed for user `3ba9eda7f6fe4ebbbc5a176eadd5ea3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:12.774507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:12.783566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:12.791990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40b14d-cd37-49c4-8d9f-471f80743aba admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.796715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.797318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.797602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.797874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.798544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.800457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.816399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:12.828746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1030/2043] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/projects => generated 401 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.829522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.845735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1014/2044] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:12.845735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.860417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835aa35-290a-459b-888f-080bd2e1e2d2 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.898658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c94e41-821d-4651-b934-48bda8cdfc50 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.904904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.905616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.905898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.906178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] PATH_INFO: `/v3/projects/d07a18118f704d6a83a13262c918df21/users/4fbbb2241f3941a6aa199ddc6ebc479c/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.908131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d07a18118f704d6a83a13262c918df21, user_id=4fbbb2241f3941a6aa199ddc6ebc479c, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.910841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.919039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.919649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.919930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.920196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.920842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:12.928847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1031/2045] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:12 2022] PUT /identity/v3/projects/d07a18118f704d6a83a13262c918df21/users/4fbbb2241f3941a6aa199ddc6ebc479c/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:12.928847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:12.929041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90901702-2c64-4b3f-9fe4-452e5b1672d6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:12.930597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:12.948833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1015/2046] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:12.950057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.950335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.950719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:12.977066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] MFA Rules not processed for user `4fbbb2241f3941a6aa199ddc6ebc479c`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:12.978267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:12.991588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c89fd9-1a4a-43f2-9f7c-cdc3f282a33d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:12.995215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:12.995838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:12.996122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:12.996389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.046058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1016/2047] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:12 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:13.046925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.052958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.054508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.064206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1032/2048] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:13.065116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.066648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.076116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:13.078107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539216d-f194-4bf7-b4c3-c8d9c7754ff3 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.083578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.084459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.084872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.085356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.095445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.096997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.108590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1033/2049] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:13.116229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c5f65-db30-40a9-b952-40cf92a9e655 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.123979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.125046 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.125785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.126672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.128934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.131551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.150687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1017/2050] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:13.154029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.164918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d763fbe-e4d3-438c-88f6-2be28a153bf9 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.166312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.171151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.171998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.172282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.172554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] PATH_INFO: `/v3/projects/e13b165cebf74770ba6abad509312442/users/6f05c24ae8ab40dd979422db72645b26/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.176640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3805f-b322-40e0-af5d-9ab2aed81631 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.180782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.181388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.181667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.182175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.192056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13b165cebf74770ba6abad509312442, user_id=6f05c24ae8ab40dd979422db72645b26, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.196230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.213080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1034/2051] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:13 2022] PUT /identity/v3/projects/e13b165cebf74770ba6abad509312442/users/6f05c24ae8ab40dd979422db72645b26/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:13.217316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.227150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:13.265539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.267986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.279020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1018/2052] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:13.306931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f59a59-3794-4580-9229-80bc503eddb0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.312554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.313238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.313587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.314089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.314833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.316492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72d5bf-d3d4-4dab-a48e-4b292bf37c1d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.328212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1035/2053] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:13.333193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.346852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50856c-2cee-4726-8e26-91c272e252a9 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.366362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.367478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.367911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.368293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] PATH_INFO: `/v3/projects/e13b165cebf74770ba6abad509312442/users/6f05c24ae8ab40dd979422db72645b26/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.371418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e13b165cebf74770ba6abad509312442, user_id=6f05c24ae8ab40dd979422db72645b26, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.376108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.403977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1019/2054] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:13 2022] PUT /identity/v3/projects/e13b165cebf74770ba6abad509312442/users/6f05c24ae8ab40dd979422db72645b26/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:13.420651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.421053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.421396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.425871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.430725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.470433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] MFA Rules not processed for user `6f05c24ae8ab40dd979422db72645b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:13.472147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.562249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673483c-a5cd-4700-abd9-56b63e2dc1e5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.567690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.568329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.568606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.568882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] PATH_INFO: `/v3/users/8051a29d0fa74a6caa886b26eedc6772` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.570641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8051a29d0fa74a6caa886b26eedc6772)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.572556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.584568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1036/2055] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:13 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:13.598806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.639744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8051a29d0fa74a6caa886b26eedc6772'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:13.641817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:13.654810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8051a29d0fa74a6caa886b26eedc6772'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:13.664137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8051a29d0fa74a6caa886b26eedc6772'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:13.665124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8051a29d0fa74a6caa886b26eedc6772'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:13.666183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a371cb2f-0511-4e76-963f-8cc577fca95e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8051a29d0fa74a6caa886b26eedc6772'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:13.690149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ca784-644d-4f55-87df-bc27a736f20b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.693343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1020/2056] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:13 2022] DELETE /identity/v3/users/8051a29d0fa74a6caa886b26eedc6772 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:13.695137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.695459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.696059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.717097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:13.718404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.742770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:13.751651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.752258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.752537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.752815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.754648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.784631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1021/2057] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:13.796008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.797498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.818913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.820445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.832172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1037/2058] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:13.864928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381065da-0b1a-4ecb-a408-f6ae19aaf8b4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.869299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.869952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.870333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.870654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.880332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:13.881794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:13.891459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1022/2059] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:14:13.923022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:13.924523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:13.936562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afdcb1-88eb-426a-81f6-80ace913dfe9 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:13.940698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:13.941311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:13.941633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:13.941909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:13.943744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:14.006783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:14.008258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:14.017991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1038/2060] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:13 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:14.168206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:14.169836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:14.200448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753995-988e-4e49-a833-32aadec20070 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:14.205719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:14.206361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:14.206696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:14.206969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] PATH_INFO: `/v3/projects/a5a4a087aa474a3da829492383570e15` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:14.208135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a4a087aa474a3da829492383570e15)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:14.210030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:14.259570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a4a087aa474a3da829492383570e15', 'request_id': 'req-8528df72-19db-42c6-b1d3-67b83aad1898'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.267147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a4a087aa474a3da829492383570e15', 'request_id': 'req-8528df72-19db-42c6-b1d3-67b83aad1898'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.271218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:14.282665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a4a087aa474a3da829492383570e15', 'request_id': 'req-8528df72-19db-42c6-b1d3-67b83aad1898'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.294214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] The token cache is being invalidate because project a5a4a087aa474a3da829492383570e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:14.294503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.295171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.296707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1023/2061] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:14 2022] DELETE /identity/v3/projects/a5a4a087aa474a3da829492383570e15 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:14.300581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:14.315405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb9a8a-551d-4249-831f-d9ca65273ec2 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:14.372629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:14.373258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:14.373537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:14.373808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] PATH_INFO: `/v3/users/ed430b3cfa5746268c2882cc833a8f7b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:14.374947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed430b3cfa5746268c2882cc833a8f7b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:14.377125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:14.417215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:14.429193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed430b3cfa5746268c2882cc833a8f7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.436547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed430b3cfa5746268c2882cc833a8f7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.444062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed430b3cfa5746268c2882cc833a8f7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.444766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed430b3cfa5746268c2882cc833a8f7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.445439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-88c2e937-d155-4999-8de7-3dfb392a173a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed430b3cfa5746268c2882cc833a8f7b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.466818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1039/2062] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:14 2022] DELETE /identity/v3/users/ed430b3cfa5746268c2882cc833a8f7b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:14.775284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:14.776833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:14.792538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528df72-19db-42c6-b1d3-67b83aad1898 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:14.871526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:14.871526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:14.871526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:14.871526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] PATH_INFO: `/v3/projects/80e2a1baf06244209dffdde2a5550538` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:14.871855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e2a1baf06244209dffdde2a5550538)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:14.871855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:14.910940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e2a1baf06244209dffdde2a5550538', 'request_id': 'req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.916232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e2a1baf06244209dffdde2a5550538', 'request_id': 'req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.923621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e2a1baf06244209dffdde2a5550538', 'request_id': 'req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.935345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] The token cache is being invalidate because project 80e2a1baf06244209dffdde2a5550538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:14.935635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.936271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-52c7c32a-ccf7-47d8-93dd-a307aa6dcd99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:14.937800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1024/2063] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:14 2022] DELETE /identity/v3/projects/80e2a1baf06244209dffdde2a5550538 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:14.943886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:14.944359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:14.944470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:14.963838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:14.965006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:15.024037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1040/2064] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:15.032224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.032511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.032787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.053433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:15.054663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:15.063375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1025/2065] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:15.067556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.077962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa540cb-0f25-4240-a8b6-de937c70d3a3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.083280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.083882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.084263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.084553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.085291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:15.085609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.087261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.101349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1041/2066] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:15.107330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.116784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65a37f7-2b28-44b2-82f8-e9b5cc1c38f6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.122756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.123449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.123693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.123966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.124598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.126500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.153436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1026/2067] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:15 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:15.157589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.167802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad5201-cac3-4541-a3fe-98bd662e231c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.222097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.222720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.222998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.223268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.223895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.225781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.247033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1042/2068] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:15 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:15.251262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.261570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d592c-e6ee-42fd-8420-850e4d7b372a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.266312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.266948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.267229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.267502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.268142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.269728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.279481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1027/2069] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:15.283083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.293364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f934b5-c0a8-40f2-bbd3-2d3bc3817390 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.297956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.298537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.298844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.299111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.299743 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.301294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91d0a1-de67-45eb-88d7-1835b3f0544e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.312425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1043/2070] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:15.316334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.326544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742fbf9-1998-4496-94be-b0f65c358b34 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.331514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.332174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.332461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.332777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] PATH_INFO: `/v3/projects/18cfdf7260934ed08e7217a27c69ecc0/users/2e51c9e1de5544628e652b2d036f9686/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.349303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cfdf7260934ed08e7217a27c69ecc0, user_id=2e51c9e1de5544628e652b2d036f9686, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.352030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.365285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1028/2071] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:15 2022] PUT /identity/v3/projects/18cfdf7260934ed08e7217a27c69ecc0/users/2e51c9e1de5544628e652b2d036f9686/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:15.377301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.377585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.377858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.405519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] MFA Rules not processed for user `2e51c9e1de5544628e652b2d036f9686`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:15.406600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:15.411863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:15.472885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1044/2072] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:15.484970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.486656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:15.505977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:15.543566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e095e4-dc7f-4ff8-98cf-aa9c74bd576b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.547216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.547790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.548069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.548340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.556919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.558567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9353f141-db32-427d-b60c-e2b0ba2c163a service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:15.567652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1029/2073] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:15.905782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:15.907546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:15.983130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19807e65-2933-44a5-9f2c-c7bbb3cf35a4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:15.987597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:15.988218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:15.988497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:15.988768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] PATH_INFO: `/v3/users/3ba9eda7f6fe4ebbbc5a176eadd5ea3e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:15.989880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ba9eda7f6fe4ebbbc5a176eadd5ea3e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:15.991787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:16.039881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ba9eda7f6fe4ebbbc5a176eadd5ea3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.046974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ba9eda7f6fe4ebbbc5a176eadd5ea3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.054523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ba9eda7f6fe4ebbbc5a176eadd5ea3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.055467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ba9eda7f6fe4ebbbc5a176eadd5ea3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.056174 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ba9eda7f6fe4ebbbc5a176eadd5ea3e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.075033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1045/2074] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:15 2022] DELETE /identity/v3/users/3ba9eda7f6fe4ebbbc5a176eadd5ea3e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:16.079759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.080204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.080557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.100974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:16.102238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:16.166167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1030/2075] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:16.176196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:16.177607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:16.210697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:16.241406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c1c490-8a6d-4d17-a33a-0e20488e7a11 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:16.245735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:16.246339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.246667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.246931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.261199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:16.263126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044f1b3-4d1b-45af-85a9-e56825b59ba3 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:16.278503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1046/2076] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:16.615517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:16.617648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:16.638835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf423c-c32d-4eed-a99d-b8cadad76abd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:16.645705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:16.646419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.646720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.646936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] PATH_INFO: `/v3/projects/1b0c83c4cd1841db9e83459c754e3417` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.648221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b0c83c4cd1841db9e83459c754e3417)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:16.650321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:16.711836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b0c83c4cd1841db9e83459c754e3417', 'request_id': 'req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.718998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b0c83c4cd1841db9e83459c754e3417', 'request_id': 'req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.733281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b0c83c4cd1841db9e83459c754e3417', 'request_id': 'req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.752155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] The token cache is being invalidate because project 1b0c83c4cd1841db9e83459c754e3417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:16.752528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.753408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5cf0ae42-65c1-4d47-a445-7d5e71d6c471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:16.755748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1031/2077] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:16 2022] DELETE /identity/v3/projects/1b0c83c4cd1841db9e83459c754e3417 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:16.761048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.761488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.761826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.802802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:16.802802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:16.871668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1047/2078] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:16.879509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.879849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.880128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.903544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:16.904377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:16.913131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1032/2079] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:16.917655 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:16.934145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae58e90-f33f-4379-9e78-be443da1cafe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:16.939530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:16.940261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.940484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.940759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.941418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:16.941730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:16.943339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:16.961711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1048/2080] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:16.965831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:16.974979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0cc895-4718-42a8-a50e-6e99194cafbd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:16.979714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:16.980334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:16.980576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:16.980856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:16.981483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:16.983476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.015076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1033/2081] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:16 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:17.016132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:17.026919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc01b4-ecc3-4a06-bb8f-fd1d38b5ae5c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:17.093443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:17.094258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.094635 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.094952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.095808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:17.098500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.153630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1049/2082] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:17 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:17.153820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:17.165828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf811f-3289-4025-a6db-5156f861adeb admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:17.171483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:17.172134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.172424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.172699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.173380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:17.175395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.187144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1034/2083] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:17.191175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:17.204208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8b4896-20c2-4f17-b05c-afc41e6adc9a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:17.210813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:17.211415 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.211693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.211957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.212605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:17.214376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1c76e-dd05-4bae-b46e-8a1fdf68f6e5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.228405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1050/2084] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:17.233635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:17.241366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:17.246629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588e1f9-8aaf-496c-89c7-95958eb88a61 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:17.253860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:17.254757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.255112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.255451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] PATH_INFO: `/v3/projects/e51ac31ebaa545f1bb4b44e35db7ebdb/users/a666cb71e05c4ad58269bcffb5d457ad/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.281600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51ac31ebaa545f1bb4b44e35db7ebdb, user_id=a666cb71e05c4ad58269bcffb5d457ad, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:17.284374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.298640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1035/2085] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:17 2022] PUT /identity/v3/projects/e51ac31ebaa545f1bb4b44e35db7ebdb/users/a666cb71e05c4ad58269bcffb5d457ad/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:17.313195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.313517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.313807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.345160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] MFA Rules not processed for user `a666cb71e05c4ad58269bcffb5d457ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:17.346550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:17.376203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:17.419304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1051/2086] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:17.431679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:17.434071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:17.495521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e4c35-8eac-48ac-95f0-98a4b3393c8d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:17.499100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:17.499702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:17.499985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:17.500262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:17.508341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:17.509770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8d5e3-8242-4e6d-8ad8-c5e2521e9a55 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:17.518376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1036/2087] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:18.046134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:18.048874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:18.134688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f397c00-e87b-468e-950e-62d29eb30158 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:18.140139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:18.140957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:18.141243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:18.141518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] PATH_INFO: `/v3/users/2f9a114987a445f9888168a01e2e6e30` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:18.142781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9a114987a445f9888168a01e2e6e30)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:18.145450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:18.223589 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9a114987a445f9888168a01e2e6e30'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.231977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9a114987a445f9888168a01e2e6e30'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.240264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9a114987a445f9888168a01e2e6e30'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.240946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9a114987a445f9888168a01e2e6e30'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.241636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9a114987a445f9888168a01e2e6e30'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.259621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1052/2088] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:18 2022] DELETE /identity/v3/users/2f9a114987a445f9888168a01e2e6e30 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:18.264127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:18.264630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:18.264723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:18.283521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:18.284721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:18.343044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1037/2089] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:18.352685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:18.354119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:18.399274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:18.410949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab7bea-4244-4eeb-8ca4-2dfeefb8f603 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:18.414738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:18.415332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:18.415613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:18.415884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:18.425788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:18.427295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:18.437433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1053/2090] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:18.697922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:18.699458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:18.713343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd1738-da03-4b28-9cbc-241d9c51a5a1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:18.718385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:18.719043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:18.719322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:18.719592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] PATH_INFO: `/v3/projects/2b6a883384f640dcb6a8db0cc8d28635` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:18.721665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6a883384f640dcb6a8db0cc8d28635)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:18.723725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:18.772367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6a883384f640dcb6a8db0cc8d28635', 'request_id': 'req-2a1fa4bb-af21-4444-881f-6688c249b11f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.778465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6a883384f640dcb6a8db0cc8d28635', 'request_id': 'req-2a1fa4bb-af21-4444-881f-6688c249b11f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.785789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6a883384f640dcb6a8db0cc8d28635', 'request_id': 'req-2a1fa4bb-af21-4444-881f-6688c249b11f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.796756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] The token cache is being invalidate because project 2b6a883384f640dcb6a8db0cc8d28635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:18.797044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.797673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.799274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1038/2091] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:18 2022] DELETE /identity/v3/projects/2b6a883384f640dcb6a8db0cc8d28635 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:18.803261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:18.813269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3623dd-1720-4527-a947-e7a709a9fe64 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:18.873908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:18.874538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:18.874847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:18.875121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] PATH_INFO: `/v3/users/4fbbb2241f3941a6aa199ddc6ebc479c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:18.876242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fbbb2241f3941a6aa199ddc6ebc479c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:18.878129 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:18.923608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fbbb2241f3941a6aa199ddc6ebc479c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.930890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fbbb2241f3941a6aa199ddc6ebc479c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.938416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fbbb2241f3941a6aa199ddc6ebc479c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.939107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fbbb2241f3941a6aa199ddc6ebc479c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.939728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a45c2961-c27f-4090-b958-caaea9c53181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fbbb2241f3941a6aa199ddc6ebc479c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:18.956877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1054/2092] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:18 2022] DELETE /identity/v3/users/4fbbb2241f3941a6aa199ddc6ebc479c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:19.251452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.253703 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:19.280266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1fa4bb-af21-4444-881f-6688c249b11f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.284654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:19.349225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.349774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.350057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.350336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] PATH_INFO: `/v3/projects/d07a18118f704d6a83a13262c918df21` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.351519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d07a18118f704d6a83a13262c918df21)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.353431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.408562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd07a18118f704d6a83a13262c918df21', 'request_id': 'req-c6b89026-530c-4728-aaef-756d1dfab2bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:19.414801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd07a18118f704d6a83a13262c918df21', 'request_id': 'req-c6b89026-530c-4728-aaef-756d1dfab2bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:19.423386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd07a18118f704d6a83a13262c918df21', 'request_id': 'req-c6b89026-530c-4728-aaef-756d1dfab2bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:19.435960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] The token cache is being invalidate because project d07a18118f704d6a83a13262c918df21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:19.436251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:19.436895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c6b89026-530c-4728-aaef-756d1dfab2bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:19.438394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1039/2093] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:19 2022] DELETE /identity/v3/projects/d07a18118f704d6a83a13262c918df21 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:19.443250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.443733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.443883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.464040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:19.465264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:19.531767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1055/2094] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:19.540501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.540762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.541040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.562446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:19.563687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:19.571494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1040/2095] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:19.575641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.585204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d88190-b606-4aaa-bc73-732175c0e8bf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.589761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.590347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.590658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.590931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.591564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:19.591863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.593431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.594868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:19.609963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1056/2096] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:14:19.610818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.620402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bd6890-46b5-426e-917c-457ff5476a96 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.625110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.625704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.625986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.626260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.626942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.628830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.654620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1041/2097] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:19.658595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.668870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaf330-8d11-44ec-afbf-ce4c86fc25c7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.723469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.724128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.724408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.724676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.725304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.727236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.760875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1057/2098] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:19.763899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.774124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77d70a-44f1-4745-932e-78a8b5f4afd7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.779228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.779811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.780086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.780354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.780987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.782548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.792649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1042/2099] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:19.796545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.806486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d6b5d-b7a3-4d38-8643-6b97edc483c6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.811145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.811731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.812014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.812290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.812928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.814544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185f3df-5386-4c6c-b13b-69abe14e4668 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.826381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1058/2100] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:19.830817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:19.841340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64235b2c-cf03-424e-98d2-b4e09f5cfe6a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:19.845860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:19.846487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.846804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.847077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] PATH_INFO: `/v3/projects/c74773d1f0144ce79c3f8aa1b5fa94a6/users/2c0c7341e0644b9f87f8bd5b3360799c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.868084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c74773d1f0144ce79c3f8aa1b5fa94a6, user_id=2c0c7341e0644b9f87f8bd5b3360799c, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:19.871005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:19.885237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1043/2101] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:19 2022] PUT /identity/v3/projects/c74773d1f0144ce79c3f8aa1b5fa94a6/users/2c0c7341e0644b9f87f8bd5b3360799c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:19.896615 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:19.896933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:19.897214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:19.924145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] MFA Rules not processed for user `2c0c7341e0644b9f87f8bd5b3360799c`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:19.925345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:19.987850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1059/2102] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:19.992383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.025994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:20.053335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26a09-0dab-445b-bbad-1590689ad7b4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.058772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.059363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.059640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.059909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.060517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.062442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.088053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1044/2103] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:19 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:20.092132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.102639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae6d83-e26e-4e85-a562-ab5e83c1efb5 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.158018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.158539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.158856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.159129 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.159770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.161660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.170223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:20.184072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1060/2104] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:20 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:20.188271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.199187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf3e6e-48e9-4911-b3ae-9269d2f4ca49 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.203766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.204395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.204675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.204946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.205585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.207219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.219193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1045/2105] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:20.223369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.234150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a27078-8c9c-4a8e-b4e2-20c280f817a5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.239070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.239541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.239853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.240149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.240821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.242446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a259ecd-234f-4a5a-8930-3e2385ee8d91 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.253942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1061/2106] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:20.258045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.268541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095418cb-d3de-4e3f-a641-4836597bfefd admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.273087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.273721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.273999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.274268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] PATH_INFO: `/v3/projects/36eb93dd8ff04d00a27b24cb5d05b88b/users/4ce5a4a3556440059a9ff74b5419b007/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.288697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36eb93dd8ff04d00a27b24cb5d05b88b, user_id=4ce5a4a3556440059a9ff74b5419b007, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.291368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.304146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1046/2107] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:20 2022] PUT /identity/v3/projects/36eb93dd8ff04d00a27b24cb5d05b88b/users/4ce5a4a3556440059a9ff74b5419b007/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:20.315086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d40916-950b-4811-a0ee-88f253871671 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.315401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d40916-950b-4811-a0ee-88f253871671 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.315702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d40916-950b-4811-a0ee-88f253871671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.343191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-72d40916-950b-4811-a0ee-88f253871671 None None] MFA Rules not processed for user `4ce5a4a3556440059a9ff74b5419b007`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:20.344383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-72d40916-950b-4811-a0ee-88f253871671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:20.415365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1062/2108] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:20.420301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.485615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f787ab7-8a39-438a-a673-dad82702cdd9 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.496982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.497607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.497891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.498166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.498925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.501048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.539637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1047/2109] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:20 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:20.543983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d40916-950b-4811-a0ee-88f253871671 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.556326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d40916-950b-4811-a0ee-88f253871671 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.632234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.634036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.634036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.634301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.635399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.638542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.661168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1063/2110] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:20 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:20.664973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.675690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b333f9-51c4-4bf2-828c-e42ec9f9f28c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.680824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.681434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.681714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.681989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.683207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.685181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.696333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1048/2111] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:20.700985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.717101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b98dac-c83e-47cc-b4ea-409d9bd606e2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.722559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.723247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.723531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.723806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] PATH_INFO: `/v3/projects/59002b44a56c4279921f62f15910d9a3/users/8ab166111e394217b343a87220643a61/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.740519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59002b44a56c4279921f62f15910d9a3, user_id=8ab166111e394217b343a87220643a61, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.743285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.756801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1064/2112] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:20 2022] PUT /identity/v3/projects/59002b44a56c4279921f62f15910d9a3/users/8ab166111e394217b343a87220643a61/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:20.760713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.770532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6b9522-88fc-4928-882e-5d1e1ef0ebba admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.819952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:20.826832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.827640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.827921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.828213 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.828856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.830496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa94c21-d4f4-4f7a-804f-3b4a35e56204 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.840731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1049/2113] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:20.844764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.855490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdb7cf-b430-4130-8c15-9541789586f7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:20.860132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:20.860773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.861051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.861322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] PATH_INFO: `/v3/projects/59002b44a56c4279921f62f15910d9a3/users/8ab166111e394217b343a87220643a61/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.863255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59002b44a56c4279921f62f15910d9a3, user_id=8ab166111e394217b343a87220643a61, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:20.865839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:20.876032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1065/2114] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:20 2022] PUT /identity/v3/projects/59002b44a56c4279921f62f15910d9a3/users/8ab166111e394217b343a87220643a61/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:20.887214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:20.887498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:20.887775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:20.914903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] MFA Rules not processed for user `8ab166111e394217b343a87220643a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:20.916129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:20.985750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1050/2115] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:20.997037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:20.998824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:21.024642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:21.056280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131fa4-9fe1-4c6e-a2d0-dfaf2cbb0d74 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:21.059931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:21.060533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:21.060793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:21.061063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:21.069762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:21.071194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:21.080810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1066/2116] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:21.120773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:21.122178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:21.133986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c59ea2-906f-442d-a755-e0775fa7e5be None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:21.138221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:21.138892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:21.139186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:21.139446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:21.197747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:21.199828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9aaf04-4e3b-4883-ab97-78815d73b7b2 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:21.209709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1051/2117] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:31.076814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=16 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:31.205156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=65 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:35.136608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.138018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.202867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f947c-7a28-4d27-a16b-e5527f310be4 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:35.207375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:35.208008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.208286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.208559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] PATH_INFO: `/v3/users/6f32f988241449bfaaee1a5406676391` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.209679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f32f988241449bfaaee1a5406676391)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:35.211654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:35.262788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f32f988241449bfaaee1a5406676391'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.271724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f32f988241449bfaaee1a5406676391'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.281498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f32f988241449bfaaee1a5406676391'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.282214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f32f988241449bfaaee1a5406676391'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.282846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f32f988241449bfaaee1a5406676391'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.301508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1067/2118] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:35 2022] DELETE /identity/v3/users/6f32f988241449bfaaee1a5406676391 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:35.305898 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.306368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.306461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.325476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:35.326704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.392505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1052/2119] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:35.402565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.403994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.466001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee4407-8ddd-4473-8c39-5372cd584eee admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:35.469659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:35.470252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.470529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.470826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.481399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:35.482910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4f9b-6da3-4b64-99dc-21a4c3ecd379 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:35.492742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1068/2120] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:35.650014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.652667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.664552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0cce-8826-41ab-a3ab-88b0092af658 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:35.669444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:35.670116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.670400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.670724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] PATH_INFO: `/v3/users/2c0c7341e0644b9f87f8bd5b3360799c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.671881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0c7341e0644b9f87f8bd5b3360799c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:35.673859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:35.721919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0c7341e0644b9f87f8bd5b3360799c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.729008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0c7341e0644b9f87f8bd5b3360799c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.738368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0c7341e0644b9f87f8bd5b3360799c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.739838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0c7341e0644b9f87f8bd5b3360799c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.741270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0c7341e0644b9f87f8bd5b3360799c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.768175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1053/2121] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:35 2022] DELETE /identity/v3/users/2c0c7341e0644b9f87f8bd5b3360799c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:35.772859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.773217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.773620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.776860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.778312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.806244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:35.808323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.826203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:35.847526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5dfcc-1010-4e3c-aca8-c90c7c52c34d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:35.852814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:35.853453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.853737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.854595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] PATH_INFO: `/v3/projects/1fa237924dec409fa915a68666dc4010` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.855743 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fa237924dec409fa915a68666dc4010)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:35.857664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:35.873716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:35.875626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1069/2122] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:35.886848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.888449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.907825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fa237924dec409fa915a68666dc4010', 'request_id': 'req-82790d50-dabc-4f69-a42e-cd306c2c3cf8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.913425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fa237924dec409fa915a68666dc4010', 'request_id': 'req-82790d50-dabc-4f69-a42e-cd306c2c3cf8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.920547 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fa237924dec409fa915a68666dc4010', 'request_id': 'req-82790d50-dabc-4f69-a42e-cd306c2c3cf8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.931846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] The token cache is being invalidate because project 1fa237924dec409fa915a68666dc4010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:35.932661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.933322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:35.935058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1054/2123] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:35 2022] DELETE /identity/v3/projects/1fa237924dec409fa915a68666dc4010 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:35.938597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:35.939892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:35.946987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2204315-59a4-4b3c-9a4c-795a30fe3fb2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:35.995083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:35.995686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:35.995970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:35.996247 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:35.997992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.004424 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82790d50-dabc-4f69-a42e-cd306c2c3cf8 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.009051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.009630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.009752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.009939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.010767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] PATH_INFO: `/v3/users/6f05c24ae8ab40dd979422db72645b26` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.011572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.011895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f05c24ae8ab40dd979422db72645b26)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.013815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.021214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1070/2124] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:36.062400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f05c24ae8ab40dd979422db72645b26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.072378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f05c24ae8ab40dd979422db72645b26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.081848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f05c24ae8ab40dd979422db72645b26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.083103 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f05c24ae8ab40dd979422db72645b26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.084740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f05c24ae8ab40dd979422db72645b26'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.108479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1055/2125] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:35 2022] DELETE /identity/v3/users/6f05c24ae8ab40dd979422db72645b26 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:36.374722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.376729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.390703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4d309-4a66-4127-bb9a-27f299f077bd service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.479205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.480722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.499091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.499603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.499951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.500139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] PATH_INFO: `/v3/projects/c74773d1f0144ce79c3f8aa1b5fa94a6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.501476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c74773d1f0144ce79c3f8aa1b5fa94a6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.503759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.520888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.536404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.575513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd90c19b-3528-4ee4-a438-f55733959e70 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.581020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c74773d1f0144ce79c3f8aa1b5fa94a6', 'request_id': 'req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.591807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c74773d1f0144ce79c3f8aa1b5fa94a6', 'request_id': 'req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.601047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c74773d1f0144ce79c3f8aa1b5fa94a6', 'request_id': 'req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.626793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] The token cache is being invalidate because project c74773d1f0144ce79c3f8aa1b5fa94a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:36.626793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.626793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.627317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1071/2126] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:36 2022] DELETE /identity/v3/projects/c74773d1f0144ce79c3f8aa1b5fa94a6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:36.629868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.631141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.654010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.654972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.655420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.655830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] PATH_INFO: `/v3/projects/e13b165cebf74770ba6abad509312442` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.657264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e13b165cebf74770ba6abad509312442)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.659658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.715435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33d4a-85bb-4f5c-8f3b-73a7436849df admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.722624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e13b165cebf74770ba6abad509312442', 'request_id': 'req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.730418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e13b165cebf74770ba6abad509312442', 'request_id': 'req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.741419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e13b165cebf74770ba6abad509312442', 'request_id': 'req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.754886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] The token cache is being invalidate because project e13b165cebf74770ba6abad509312442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:36.755283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.756016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-96d48d0e-e31b-4480-a240-6bfe2bb1a7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.757680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1056/2127] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:36 2022] DELETE /identity/v3/projects/e13b165cebf74770ba6abad509312442 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:36.770779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295759-348a-4e58-927c-5ee2f0f4a249 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.770779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295759-348a-4e58-927c-5ee2f0f4a249 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.770779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a295759-348a-4e58-927c-5ee2f0f4a249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.785595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-7a295759-348a-4e58-927c-5ee2f0f4a249 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:36.787910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7a295759-348a-4e58-927c-5ee2f0f4a249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.794964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.795603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.795887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.796161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] PATH_INFO: `/v3/users/4ce5a4a3556440059a9ff74b5419b007` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.797312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce5a4a3556440059a9ff74b5419b007)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.799297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.842498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.851984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1057/2128] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:36.854001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce5a4a3556440059a9ff74b5419b007'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.859963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.861721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce5a4a3556440059a9ff74b5419b007'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.862914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.863530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.866188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.870724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce5a4a3556440059a9ff74b5419b007'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.871957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce5a4a3556440059a9ff74b5419b007'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.872851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce5a4a3556440059a9ff74b5419b007'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:36.883955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:36.885674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.891968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1072/2129] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:36 2022] DELETE /identity/v3/users/4ce5a4a3556440059a9ff74b5419b007 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:36.895002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1058/2130] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:36.897645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.959602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a2514-cc1f-439d-a13b-a9992b6eb43b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.964385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.964989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.965270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.965538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.966181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:36.966485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.968097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.980554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1073/2131] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:14:36.984991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.994313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd21bb6-e6ce-4982-8fab-c4a7a871011b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.999077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.999680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.999962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.000246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.000865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.003297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.033430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1059/2132] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:36 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:37.034704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.045553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009fa41b-214b-41f2-a05d-04042c5c094f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.104773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.105632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.106040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.106428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.108042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.111241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.140524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1074/2133] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:37 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:37.141944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.159975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad0c42-7fb4-4e6e-aef6-79c4d34fbee4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.165151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.165670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.165949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.166220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.167449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.169029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.176143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.178029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:37.180069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1060/2134] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:37.184450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.195268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a9a893-8f95-4be8-81d8-6ef77f4366c6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.196070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac8c3e-c556-46b7-a335-ee7e0fa4cde8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.200191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.200582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.200796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.201209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.201490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.201772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.202060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.202174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.202336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] PATH_INFO: `/v3/projects/36eb93dd8ff04d00a27b24cb5d05b88b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.203695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36eb93dd8ff04d00a27b24cb5d05b88b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.203810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.205572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.215441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1061/2135] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:14:37.219524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.229671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e57c87-fa60-4159-b820-aeaaf5867001 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.234416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.235139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.235421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.235699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] PATH_INFO: `/v3/projects/e5bd389686bd4de7a37ff25afe93c11a/users/9f8f5fefe88e46338a57cac053799e4f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.251187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5bd389686bd4de7a37ff25afe93c11a, user_id=9f8f5fefe88e46338a57cac053799e4f, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.253909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee0a0d4-9fa4-4b5e-87b6-627adff76d45 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.258053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36eb93dd8ff04d00a27b24cb5d05b88b', 'request_id': 'req-75c5bff4-e303-465f-88d2-aba9ebc65806'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.263264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:37.264577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36eb93dd8ff04d00a27b24cb5d05b88b', 'request_id': 'req-75c5bff4-e303-465f-88d2-aba9ebc65806'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.267493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1062/2136] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:37 2022] PUT /identity/v3/projects/e5bd389686bd4de7a37ff25afe93c11a/users/9f8f5fefe88e46338a57cac053799e4f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:37.279322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36eb93dd8ff04d00a27b24cb5d05b88b', 'request_id': 'req-75c5bff4-e303-465f-88d2-aba9ebc65806'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.279580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.280039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.280528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.292079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] The token cache is being invalidate because project 36eb93dd8ff04d00a27b24cb5d05b88b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:37.292456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.293115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.294661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1075/2137] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:37 2022] DELETE /identity/v3/projects/36eb93dd8ff04d00a27b24cb5d05b88b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:37.298356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.299760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:37.308962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] MFA Rules not processed for user `9f8f5fefe88e46338a57cac053799e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:37.310139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:37.364092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5bff4-e303-465f-88d2-aba9ebc65806 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.374077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1063/2138] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:37.385370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.386763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:37.429606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.430238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.430801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.431075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] PATH_INFO: `/v3/users/8ab166111e394217b343a87220643a61` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.432300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab166111e394217b343a87220643a61)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.434192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.443590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7340b65-fc0d-4b0b-8121-1d49cf8c19aa None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.448293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.448902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.449186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.449461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.459572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.461341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:37.471931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1064/2139] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:14:37.482948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab166111e394217b343a87220643a61'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.490299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab166111e394217b343a87220643a61'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.498385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab166111e394217b343a87220643a61'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.499199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab166111e394217b343a87220643a61'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.499831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-3ddbcfd7-b743-4e1a-9a42-c8733d024bff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab166111e394217b343a87220643a61'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:37.517971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1076/2140] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:37 2022] DELETE /identity/v3/users/8ab166111e394217b343a87220643a61 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:37.826818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:37.840267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be4132-473f-4f34-a160-7835bb269d35 service glance] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:37.858958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:37.926390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:37.927112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:37.927405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:37.927681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] PATH_INFO: `/v3/projects/59002b44a56c4279921f62f15910d9a3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:37.940394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59002b44a56c4279921f62f15910d9a3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:37.943409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.016677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59002b44a56c4279921f62f15910d9a3', 'request_id': 'req-53e03aa7-bd5a-4d79-98e1-e519b5de2755'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:38.022653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59002b44a56c4279921f62f15910d9a3', 'request_id': 'req-53e03aa7-bd5a-4d79-98e1-e519b5de2755'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:38.035956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59002b44a56c4279921f62f15910d9a3', 'request_id': 'req-53e03aa7-bd5a-4d79-98e1-e519b5de2755'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:38.060094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] The token cache is being invalidate because project 59002b44a56c4279921f62f15910d9a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:38.060390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:38.061259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-53e03aa7-bd5a-4d79-98e1-e519b5de2755 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:38.062936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1065/2141] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:37 2022] DELETE /identity/v3/projects/59002b44a56c4279921f62f15910d9a3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:38.068915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.069219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.069492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.089726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:38.091038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:38.096421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:38.149781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1077/2142] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:38.158595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.158889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.159158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.179414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:38.180869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:38.189158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1066/2143] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:38.193239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.203281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1d033e-31e0-4fa9-9fd1-418d3e0ebb71 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.208071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.208666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.208950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.209225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.209898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:38.210204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.211826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.224283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1078/2144] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:38.228526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.238227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad612de-218a-40f0-b0d5-17cd5b88833f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.242821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.243406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.243680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.243949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.244606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.246480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.272355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1067/2145] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/projects => generated 381 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:38.276543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.287257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5ad9e-7774-405e-8af1-08e9c57aec63 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.342382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.343019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.343299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.343569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.344185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.346069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.366578 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1079/2146] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:38.370759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.380920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b090e7-4805-432b-9915-b920d7212f64 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.385975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.386723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.387002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.387273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.387916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.389492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.399448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1068/2147] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:38.403701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.413535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a74ef5-5ecc-4078-908f-c63b5978d470 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.418203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.418894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.419178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.419448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.421114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.421765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c832e-e185-4cca-96bd-c209a5cf1122 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.432553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1080/2148] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:38.436601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.446549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61768614-9f28-4f71-a612-9390715c880f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.451707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.452344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.452628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.452902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] PATH_INFO: `/v3/projects/1c06382bfd8f4f12a3209f4f5aaff1e5/users/3691efb19e8e44c394ca31cc0aa4abf7/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.467843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c06382bfd8f4f12a3209f4f5aaff1e5, user_id=3691efb19e8e44c394ca31cc0aa4abf7, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.472746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.483165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1069/2149] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:38 2022] PUT /identity/v3/projects/1c06382bfd8f4f12a3209f4f5aaff1e5/users/3691efb19e8e44c394ca31cc0aa4abf7/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:38.495517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.495764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.496039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.523151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] MFA Rules not processed for user `3691efb19e8e44c394ca31cc0aa4abf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:38.524346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:38.586512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1081/2150] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:38.591599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.637144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:38.653718 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed696dcb-dd3a-4b19-8c52-7034779d491e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.658328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.658957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.659238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.659508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.660124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.662027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.687494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1070/2151] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:38.691646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.702290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd17425-cf86-4a7a-b4cb-10a752fe95a6 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.705848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:38.759998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.761315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.761571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.761900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.762556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.764548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.792446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1082/2152] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:38 2022] POST /identity/v3/users => generated 298 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:38.796335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.807679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73265ea6-db00-4173-8e35-ffc981dbbd78 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.813399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.814006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.814287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.814560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.815232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.816820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.827734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1071/2153] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:38.832017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.841787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b2722-b4df-4198-8792-b4b9e576cece admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.846369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.847126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.847403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.847674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] PATH_INFO: `/v3/projects/3a4dac39002542ffba39231585e3befb/users/f721ff985fa04c6bae50c45ed454ee59/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.868027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4dac39002542ffba39231585e3befb, user_id=f721ff985fa04c6bae50c45ed454ee59, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.871381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.884214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1083/2154] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:38 2022] PUT /identity/v3/projects/3a4dac39002542ffba39231585e3befb/users/f721ff985fa04c6bae50c45ed454ee59/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:38.888820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.899687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55350504-f8fc-46c6-932d-aca77a8336a4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.956585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.957197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.957482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.957757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.958397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.960039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008e5236-cdd2-484a-9d34-578e38559352 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:38.969601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1072/2155] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:38.973768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:38.984497 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d14b79-b924-4c10-807e-e12d8aa48416 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:38.989146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:38.989814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:38.990093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:38.990366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] PATH_INFO: `/v3/projects/3a4dac39002542ffba39231585e3befb/users/f721ff985fa04c6bae50c45ed454ee59/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:38.992324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a4dac39002542ffba39231585e3befb, user_id=f721ff985fa04c6bae50c45ed454ee59, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:38.995653 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.005590 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1084/2156] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:38 2022] PUT /identity/v3/projects/3a4dac39002542ffba39231585e3befb/users/f721ff985fa04c6bae50c45ed454ee59/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:39.015612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.015860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.016135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.043215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] MFA Rules not processed for user `f721ff985fa04c6bae50c45ed454ee59`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:39.044401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.126404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1073/2157] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:39.135795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:39.137532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.194763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf85bc-227d-4b36-9ded-8ca88bdde087 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:39.198372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:39.198981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.199269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.199538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.208358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:39.209783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.219531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1085/2158] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:39.290732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:39.292695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.304552 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fc821-4dac-4338-98a7-f6c27ac79fa7 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:39.309458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:39.310155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.310601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.310989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.350261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:39.369802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:39.371374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3530d-89fb-4e4a-b964-51dbd0044439 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.380116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1074/2159] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:39.641469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:39.704307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c79c5e-bb88-411a-8050-fa379ccaecd1 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:39.709153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:39.709748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.710033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.710308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] PATH_INFO: `/v3/users/3691efb19e8e44c394ca31cc0aa4abf7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.711483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3691efb19e8e44c394ca31cc0aa4abf7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:39.713391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.716094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:39.762702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3691efb19e8e44c394ca31cc0aa4abf7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:39.770960 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3691efb19e8e44c394ca31cc0aa4abf7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:39.781145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3691efb19e8e44c394ca31cc0aa4abf7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:39.781821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3691efb19e8e44c394ca31cc0aa4abf7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:39.782461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3691efb19e8e44c394ca31cc0aa4abf7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:39.801145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1086/2160] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:39 2022] DELETE /identity/v3/users/3691efb19e8e44c394ca31cc0aa4abf7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:39.806051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.806336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.806636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.825281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:39.826476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.886648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1075/2161] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:39.896332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:39.897844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.972052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0af3c-562b-43a6-a8f1-5f4fcf4c1cbf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:39.975731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:39.976340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.976625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.976900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.986245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:39.987707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.997082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1087/2162] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:40.287540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:40.290009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:40.302950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cbaf71-4807-42fe-afdb-8d6a9a9ba2e6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:40.308770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:40.309533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:40.309909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:40.310275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] PATH_INFO: `/v3/projects/1c06382bfd8f4f12a3209f4f5aaff1e5` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:40.311712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c06382bfd8f4f12a3209f4f5aaff1e5)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:40.313695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:40.381215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c06382bfd8f4f12a3209f4f5aaff1e5', 'request_id': 'req-dd156c25-a122-4772-9f7e-6d0de3b215c3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.387698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c06382bfd8f4f12a3209f4f5aaff1e5', 'request_id': 'req-dd156c25-a122-4772-9f7e-6d0de3b215c3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.395522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c06382bfd8f4f12a3209f4f5aaff1e5', 'request_id': 'req-dd156c25-a122-4772-9f7e-6d0de3b215c3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.407285 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] The token cache is being invalidate because project 1c06382bfd8f4f12a3209f4f5aaff1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:40.407574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.408249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.409897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1076/2163] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:40 2022] DELETE /identity/v3/projects/1c06382bfd8f4f12a3209f4f5aaff1e5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:40.413855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:40.427433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9aca23-fa3a-44f2-b978-218841473ecd service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:40.484397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:40.485116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:40.485453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:40.485765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] PATH_INFO: `/v3/users/f721ff985fa04c6bae50c45ed454ee59` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:40.487142 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f721ff985fa04c6bae50c45ed454ee59)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:40.489301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:40.517292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:40.537872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f721ff985fa04c6bae50c45ed454ee59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.545341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f721ff985fa04c6bae50c45ed454ee59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.553037 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f721ff985fa04c6bae50c45ed454ee59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.553784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f721ff985fa04c6bae50c45ed454ee59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.554408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ca337792-06d2-4cb5-b346-29ec2ad9452c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f721ff985fa04c6bae50c45ed454ee59'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.574022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1088/2164] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:40 2022] DELETE /identity/v3/users/f721ff985fa04c6bae50c45ed454ee59 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:40.854446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:40.854446 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:40.871324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd156c25-a122-4772-9f7e-6d0de3b215c3 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:40.928918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:40.929660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:40.929780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:40.930052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] PATH_INFO: `/v3/projects/3a4dac39002542ffba39231585e3befb` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:40.931515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a4dac39002542ffba39231585e3befb)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:40.933958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:40.936227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:40.987015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a4dac39002542ffba39231585e3befb', 'request_id': 'req-937ce03a-b3b3-46cd-b84a-c06706ea6f97'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:40.993532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a4dac39002542ffba39231585e3befb', 'request_id': 'req-937ce03a-b3b3-46cd-b84a-c06706ea6f97'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:41.004097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a4dac39002542ffba39231585e3befb', 'request_id': 'req-937ce03a-b3b3-46cd-b84a-c06706ea6f97'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:41.018303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] The token cache is being invalidate because project 3a4dac39002542ffba39231585e3befb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:41.018755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:41.023353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-937ce03a-b3b3-46cd-b84a-c06706ea6f97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:41.026618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1077/2165] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:40 2022] DELETE /identity/v3/projects/3a4dac39002542ffba39231585e3befb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:41.033509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.033509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.033509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.054033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:41.055269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.133997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1089/2166] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:41.137342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.137744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.138122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.162556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:41.164092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.177689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.190429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e7502-d9c0-41ce-9f9f-3f0737c7e5fc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.193869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1078/2167] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:41.200397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074270c-efae-4498-97af-47e01845480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.201188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074270c-efae-4498-97af-47e01845480e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.201709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074270c-efae-4498-97af-47e01845480e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.207390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074270c-efae-4498-97af-47e01845480e admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.209147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074270c-efae-4498-97af-47e01845480e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:41.209795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074270c-efae-4498-97af-47e01845480e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.214393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074270c-efae-4498-97af-47e01845480e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.234650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1090/2168] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:41.235631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.245838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb3ddc-98d1-4b12-966f-d6f7f755dee4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.251051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.251936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.252418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.252881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.253742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.255723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.288009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1079/2169] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:41.289196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074270c-efae-4498-97af-47e01845480e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.300683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074270c-efae-4498-97af-47e01845480e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.391580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.392203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.392485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.392759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.393404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.395456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.421463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1091/2170] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:14:41.427803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.443287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64695a55-8c0b-4225-9fa3-6e7da0aac926 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.453297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.453931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.454605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.454906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.455585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.457376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.473748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1080/2171] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:41.478028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.492695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5a857-8c95-40ed-ae93-f8a5a9daa7ac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.503568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.504236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.504559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.504841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.505548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.507552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f247a97-6359-40ed-9194-6f971fd93acd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.520286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1092/2172] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:41.524263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.535766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89a76c-3396-41e7-afb2-ebfd5ed42899 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.540869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.541663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.542041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.542381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] PATH_INFO: `/v3/projects/84637182526541fb9717285ed49d48a3/users/750eca6669914c09bbd1aeb34e3bfc1b/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.561062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84637182526541fb9717285ed49d48a3, user_id=750eca6669914c09bbd1aeb34e3bfc1b, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.564480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.593128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1081/2173] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:41 2022] PUT /identity/v3/projects/84637182526541fb9717285ed49d48a3/users/750eca6669914c09bbd1aeb34e3bfc1b/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:14:41.604419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.604716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.604995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.629618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:41.643233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] MFA Rules not processed for user `750eca6669914c09bbd1aeb34e3bfc1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:41.644927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.744253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1093/2174] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:41.756221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.758012 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.864975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b96dd-f1fe-446d-aa13-3a2058ced5d2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.870809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:41.870809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.871460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.872064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.882040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:41.883527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:41.898068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1082/2175] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:44.671137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:44.672695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:44.752320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942f7cf-0626-4625-9943-a5fb1cd2a1be None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:44.756863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:44.757484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:44.757770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:44.758045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] PATH_INFO: `/v3/users/a59246460c1846ac82200f4516f2507f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:44.759308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a59246460c1846ac82200f4516f2507f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:44.761259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:44.806307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a59246460c1846ac82200f4516f2507f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:44.813121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a59246460c1846ac82200f4516f2507f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:44.820537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a59246460c1846ac82200f4516f2507f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:44.821223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a59246460c1846ac82200f4516f2507f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:44.821838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c109289a-f581-4a5b-bb3e-505e59fe00de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a59246460c1846ac82200f4516f2507f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:44.839298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1094/2176] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:14:44 2022] DELETE /identity/v3/users/a59246460c1846ac82200f4516f2507f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:45.013051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.024842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2726b4c-feff-4ae5-a8af-8b28fb251362 service glance] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.067468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:45.100500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.101221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.101572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.101913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] PATH_INFO: `/v3/projects/450e17cbc2104fcba833fc48ada83cda` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.103157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450e17cbc2104fcba833fc48ada83cda)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.105219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.153248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450e17cbc2104fcba833fc48ada83cda', 'request_id': 'req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:45.158989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450e17cbc2104fcba833fc48ada83cda', 'request_id': 'req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:45.167002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450e17cbc2104fcba833fc48ada83cda', 'request_id': 'req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:45.178548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] The token cache is being invalidate because project 450e17cbc2104fcba833fc48ada83cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:14:45.178863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:45.179501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-d3f49606-3176-4225-bfd5-4e8ce04c9eb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:14:45.181226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1083/2177] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:14:45 2022] DELETE /identity/v3/projects/450e17cbc2104fcba833fc48ada83cda => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:45.186434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.186685 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.186968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.206707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:45.207949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:45.220374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:45.267394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1095/2178] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:45.274926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.275216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.275496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.295156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:45.296357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:45.304599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1084/2179] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:45.308941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.318491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3251c-c256-4880-bf6e-6e8621fa911b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.323618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.324348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.324498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.324757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.325404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:14:45.325707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.327326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.343140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1096/2180] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:14:45.344348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.353414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6c645-611f-454c-9be6-8d0e378c6bdb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.357957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.358541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.358848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.359116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.359732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.361603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.388452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1085/2181] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:14:45 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:45.392525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.402698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8d7a8-644f-49fb-b01e-028935f23079 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.457071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.457666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.457942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.458215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.458939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.460807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.484607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1097/2182] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:14:45 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:14:45.488635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.501252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03163d74-0278-4210-8a02-e150abbd3e39 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.506131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.506784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.507070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.507347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.507998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.509592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.523512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1086/2183] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:45.524723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.535258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56c608-fc02-413a-bccf-324f3d99a671 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.553254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.553883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.554168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.554461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.555132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.557484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978face-d34d-4d0c-9b0a-e362645baa75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.569061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1098/2184] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:14:45.573560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.584443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c22477-c9a0-4782-874c-d1db81cd7a97 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.591170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.591830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.592153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.592426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] PATH_INFO: `/v3/projects/0a19aabba33c4e0ab7b3f2d9d58ec621/users/1f000354b524462bb66812138c6e4695/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.607949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a19aabba33c4e0ab7b3f2d9d58ec621, user_id=1f000354b524462bb66812138c6e4695, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.610709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.623634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1087/2185] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:14:45 2022] PUT /identity/v3/projects/0a19aabba33c4e0ab7b3f2d9d58ec621/users/1f000354b524462bb66812138c6e4695/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:14:45.628665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de17ad-10e5-418b-ae1c-acc0a62724ab None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.628954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de17ad-10e5-418b-ae1c-acc0a62724ab None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.629226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21de17ad-10e5-418b-ae1c-acc0a62724ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.649714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-21de17ad-10e5-418b-ae1c-acc0a62724ab None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:45.651465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-21de17ad-10e5-418b-ae1c-acc0a62724ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:45.708541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1099/2186] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:45.718489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:45.720221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:45.780246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b07eeac-c797-4639-84ce-66b718e50b38 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.784402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.784969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.785254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.785529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.795081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.796514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.801414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:45.807234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1088/2187] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:50.643690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c20b1-aaa2-467e-ae2e-3c818f6042ea None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:50.644186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c20b1-aaa2-467e-ae2e-3c818f6042ea None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:50.644278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c20b1-aaa2-467e-ae2e-3c818f6042ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:50.679523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-519c20b1-aaa2-467e-ae2e-3c818f6042ea None None] MFA Rules not processed for user `1f000354b524462bb66812138c6e4695`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:50.681175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-519c20b1-aaa2-467e-ae2e-3c818f6042ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:50.696668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:50.761258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1100/2188] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:50.775377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:50.776900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:50.872174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16832308-2e05-411d-aa4b-15a35088c0b5 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:50.878528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:50.879179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:50.879461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:50.879727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:50.888810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:50.890443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:50.901615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1089/2189] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:52.236973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676db08-5b30-436b-adb6-b4268040f1e2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:52.237813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676db08-5b30-436b-adb6-b4268040f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:52.238102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676db08-5b30-436b-adb6-b4268040f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:52.260076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b676db08-5b30-436b-adb6-b4268040f1e2 None None] MFA Rules not processed for user `1f000354b524462bb66812138c6e4695`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:52.261852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b676db08-5b30-436b-adb6-b4268040f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:52.268841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1101/2190] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:14:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:00.896099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=34 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:02.252900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=24 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:05.090056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:05.091568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:05.105977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cdb8d-9612-4954-bfad-f3e18c6b6e9a service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:05.110505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:05.111161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:05.111616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:05.111890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] PATH_INFO: `/v3/users/a666cb71e05c4ad58269bcffb5d457ad` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:05.112999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a666cb71e05c4ad58269bcffb5d457ad)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:05.115107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:05.170591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a666cb71e05c4ad58269bcffb5d457ad'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.177731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a666cb71e05c4ad58269bcffb5d457ad'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.186199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a666cb71e05c4ad58269bcffb5d457ad'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.186841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a666cb71e05c4ad58269bcffb5d457ad'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.187450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a666cb71e05c4ad58269bcffb5d457ad'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.217969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:05.218419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:05.218514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:05.222925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1090/2191] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:05 2022] DELETE /identity/v3/users/a666cb71e05c4ad58269bcffb5d457ad => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:05.241668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:05.242944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:05.336366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1102/2192] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:05.348529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:05.350344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:05.466973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032d3d8-7e32-4573-8399-a2cc2a51c68e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:05.473847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:05.475045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:05.475497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:05.475954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:05.492089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:05.493542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57a8bc-2aac-4702-b50e-7e058fe4e3be service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:05.503623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1091/2193] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:05.809197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:05.812386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:05.826542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcb5e5-5297-41bf-a013-2f087c02c7b8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:05.831770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:05.832632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:05.833056 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:05.833472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] PATH_INFO: `/v3/projects/e51ac31ebaa545f1bb4b44e35db7ebdb` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:05.834838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51ac31ebaa545f1bb4b44e35db7ebdb)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:05.837109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:05.914961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51ac31ebaa545f1bb4b44e35db7ebdb', 'request_id': 'req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.921341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51ac31ebaa545f1bb4b44e35db7ebdb', 'request_id': 'req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.929956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51ac31ebaa545f1bb4b44e35db7ebdb', 'request_id': 'req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.947134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] The token cache is being invalidate because project e51ac31ebaa545f1bb4b44e35db7ebdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:05.947452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.948663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d5c8a3c0-9f44-48c7-8c24-93288cb61c32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:05.950289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1103/2194] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:05 2022] DELETE /identity/v3/projects/e51ac31ebaa545f1bb4b44e35db7ebdb => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:05.955518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:05.955806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:05.956085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:05.976505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:05.977778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:06.042149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1092/2195] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:06.050032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20b591-0519-4537-85db-93d084fffffd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.050389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20b591-0519-4537-85db-93d084fffffd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.050610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20b591-0519-4537-85db-93d084fffffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.075059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-eb20b591-0519-4537-85db-93d084fffffd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:06.076265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-eb20b591-0519-4537-85db-93d084fffffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:06.084773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1104/2196] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:06.089088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.099342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c21c2-f7ba-43d3-925d-354c7f31a808 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.104762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.105618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.105931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.106238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.106962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:15:06.107265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.108860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.126375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1093/2197] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:15:06.127786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20b591-0519-4537-85db-93d084fffffd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.138039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20b591-0519-4537-85db-93d084fffffd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.143833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.144443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.144721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.145043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.146254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.148999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.180396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1105/2198] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:06 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:06.185089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.190172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:06.197027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5a557-60b9-485f-9e48-657bb1a9b88e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.266037 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.266701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.267006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.267274 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.267903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.269872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.290509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1094/2199] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:06 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:06.295133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.305963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378dcb04-8614-49c0-a760-674d65754300 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.311435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.312051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.312303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.312571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.313223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.314840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.328498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1106/2200] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:06.329208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.342354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daaae56-3867-42c1-886d-382662ac8517 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.348517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.349250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.349448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.349704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.350829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.352571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9565f-bfc9-4545-a55f-ccca222bfaed admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.365440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1095/2201] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:06.369626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.379804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb2050-4859-4dd0-afbb-ff2021e73abf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.385232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.385893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.386172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.386682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] PATH_INFO: `/v3/projects/79389800e0ea49d3b5abfbefe17960ac/users/7ebc052a3cb6400a961c1b9d9bec83d6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.402066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79389800e0ea49d3b5abfbefe17960ac, user_id=7ebc052a3cb6400a961c1b9d9bec83d6, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.404732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.418130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1107/2202] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:06 2022] PUT /identity/v3/projects/79389800e0ea49d3b5abfbefe17960ac/users/7ebc052a3cb6400a961c1b9d9bec83d6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:06.423312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.423644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.423970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.442667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:06.443870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:06.510924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1096/2203] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:06.520830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:06.523437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:06.550963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:06.595448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b7f575-bc35-40d9-a0f1-371721e80bc5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:06.599099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:06.599698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:06.599972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:06.600239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:06.613641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:06.615106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6885c9-02b2-48bd-946d-fbc43a1ffb1c service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:06.642974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1108/2204] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:07.043987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:07.046611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:07.062837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494ade8-c406-463c-b77e-33c52a2a78c0 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:07.068507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:07.069310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:07.069733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:07.070189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] PATH_INFO: `/v3/users/2e51c9e1de5544628e652b2d036f9686` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:07.071913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e51c9e1de5544628e652b2d036f9686)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:07.074243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:07.126455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e51c9e1de5544628e652b2d036f9686'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.134124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e51c9e1de5544628e652b2d036f9686'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.141929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e51c9e1de5544628e652b2d036f9686'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.142777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e51c9e1de5544628e652b2d036f9686'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.143343 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e51c9e1de5544628e652b2d036f9686'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.154984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:07.165422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1097/2205] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:07 2022] DELETE /identity/v3/users/2e51c9e1de5544628e652b2d036f9686 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:07.170790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:07.171085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:07.171413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:07.191928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:07.193121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:07.254706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1109/2206] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:07.264775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:07.266255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:07.342335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d197f8-0092-4461-9d6e-096322700ca5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:07.346669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:07.346920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:07.348013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:07.348013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:07.358112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:07.359570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff33b0-7877-4306-aed4-e5eaf5b716c2 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:07.368802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1098/2207] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:07.679853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:07.683702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:07.703641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e6e15-4a18-4760-9899-4dd3ed1493aa None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:07.709599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:07.710300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:07.710824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:07.711452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] PATH_INFO: `/v3/projects/18cfdf7260934ed08e7217a27c69ecc0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:07.712681 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cfdf7260934ed08e7217a27c69ecc0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:07.715292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:07.878954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cfdf7260934ed08e7217a27c69ecc0', 'request_id': 'req-8178dd4b-610f-4a33-911e-452fa78ca5c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.885659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cfdf7260934ed08e7217a27c69ecc0', 'request_id': 'req-8178dd4b-610f-4a33-911e-452fa78ca5c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.899533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cfdf7260934ed08e7217a27c69ecc0', 'request_id': 'req-8178dd4b-610f-4a33-911e-452fa78ca5c3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.927033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] The token cache is being invalidate because project 18cfdf7260934ed08e7217a27c69ecc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:07.927647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.928917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8178dd4b-610f-4a33-911e-452fa78ca5c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:07.946481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1110/2208] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:07 2022] DELETE /identity/v3/projects/18cfdf7260934ed08e7217a27c69ecc0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:07.946693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:07.947375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:07.947375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:07.977554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:07.980107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:08.069714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1099/2209] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:08.077877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.078162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.078434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.112848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:08.114989 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:08.126510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1111/2210] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:08.131127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.146226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0876-26fe-40b0-91f7-0dabdcb51aa4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.152027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.152643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.152922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.153255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.153994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:15:08.154302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.155981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.168916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1100/2211] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:15:08.172997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.185849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2096b4-ec2c-484a-a4f3-46de8b15291c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.191766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.192390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.192680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.193301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.193945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.197067 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.259082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1112/2212] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:08 2022] POST /identity/v3/projects => generated 369 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:08.261704 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.272904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a473-06b4-4cce-aea2-b83f809552df admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.378819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.379385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.379866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.380298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.381220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.383308 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.415594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1101/2213] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:08 2022] POST /identity/v3/users => generated 285 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:08.419545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.436226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d6ec-e9d1-4383-88c8-4eb61dfe0f71 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.445671 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.446449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.448029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.448473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.449427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.451611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.478771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1113/2214] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:08.483093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.496897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5b924-4b6a-45ae-868e-30dcdb6e80a6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.501737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.502364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.502671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.502943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.503604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.505579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66ac2d9-8e54-43c1-bb52-47816fb99065 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.521051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1102/2215] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:08.522372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.542524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc7c89-820a-4670-9da0-e68f1027b37b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.549892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.551424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.551892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.552360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] PATH_INFO: `/v3/projects/575943d6b15445c1b84708e40f430196/users/db379afd44f8450c938ad721f5f4423b/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.555367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:08.576580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575943d6b15445c1b84708e40f430196, user_id=db379afd44f8450c938ad721f5f4423b, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.580315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.599605 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1114/2216] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:08 2022] PUT /identity/v3/projects/575943d6b15445c1b84708e40f430196/users/db379afd44f8450c938ad721f5f4423b/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:08.613065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44208-3079-4d23-ae3e-0eaa9b446b6e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.613363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44208-3079-4d23-ae3e-0eaa9b446b6e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.613769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44208-3079-4d23-ae3e-0eaa9b446b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.622458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:08.669074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-afb44208-3079-4d23-ae3e-0eaa9b446b6e None None] MFA Rules not processed for user `db379afd44f8450c938ad721f5f4423b`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:08.670592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-afb44208-3079-4d23-ae3e-0eaa9b446b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:08.783067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1103/2217] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:08.791483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:08.793642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:08.893376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b857b-a3bd-4c40-84f3-e48730456ff2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:08.912810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:08.913422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:08.913713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:08.913972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:08.923533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:08.925133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:08.946397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1115/2218] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:11.847635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:11.848093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:11.848367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:11.903040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] MFA Rules not processed for user `7ebc052a3cb6400a961c1b9d9bec83d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:11.904252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:11.989771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1104/2219] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:11.992396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:11.993840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:12.063158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b4ad7-bb51-438b-b4a6-c223698c98ea service glance] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.067990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.068608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.068885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.069154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.069794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.071785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.102745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1116/2220] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:11 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:12.104394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:12.115683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50703588-0064-49b0-8c7d-6085dcdb33dd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.188325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.189243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.189622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.189904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.190540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.192504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.233642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1105/2221] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:12 2022] POST /identity/v3/users => generated 296 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:12.237527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:12.249469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ec3d9-91f0-4025-9c66-c5f21c847d26 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.254235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.254925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.255208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.255482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.256145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.257785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.269255 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1117/2222] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:12.273495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:12.284135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc9851-49b1-4a87-939d-a2e5af6e0a81 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.288834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.289522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.289830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.290027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] PATH_INFO: `/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01/users/007a5c65ed4a48648b55d27c6ff12421/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.301279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:12.312679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1b7764ba1243a8ae79974d4fa0da01, user_id=007a5c65ed4a48648b55d27c6ff12421, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.313800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.343403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1106/2223] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:12 2022] PUT /identity/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01/users/007a5c65ed4a48648b55d27c6ff12421/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:15:12.346040 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:12.360095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335882d-5157-46ec-aeec-d382d894c91a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.441930 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.442557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.442876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.443152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.443812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.445425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e336f1-72c2-4fa8-9966-38acdb8dfed7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.446903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:12.461257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1118/2224] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:12.462565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:12.483617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7048c416-7a26-4166-bcb6-9fa326c49998 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:12.489929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:12.490694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:12.491380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:12.491652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] PATH_INFO: `/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01/users/007a5c65ed4a48648b55d27c6ff12421/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:12.493844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d1b7764ba1243a8ae79974d4fa0da01, user_id=007a5c65ed4a48648b55d27c6ff12421, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:12.498181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:12.520590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1107/2225] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:12 2022] PUT /identity/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01/users/007a5c65ed4a48648b55d27c6ff12421/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:18.622282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:18.622894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:18.623208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:18.667446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] MFA Rules not processed for user `007a5c65ed4a48648b55d27c6ff12421`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:18.669632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:18.760776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1119/2226] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:18.775193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:18.777509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:18.867816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef2e26-6b5b-4996-b2dc-0c490169a9df admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:18.875064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:18.875064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:18.875064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:18.875428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:18.942545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:18.944165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e632530-1fe9-4972-b2ce-d45a1f702d0d service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:18.954789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1108/2227] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:27.683619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:27.685088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:27.754843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36828819-79f7-48f5-8dfb-1039d2238acf None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:27.759765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:27.760436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:27.760698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:27.760997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] PATH_INFO: `/v3/users/750eca6669914c09bbd1aeb34e3bfc1b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:27.762113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=750eca6669914c09bbd1aeb34e3bfc1b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:27.764050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:27.813379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750eca6669914c09bbd1aeb34e3bfc1b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:27.820713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750eca6669914c09bbd1aeb34e3bfc1b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:27.828383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750eca6669914c09bbd1aeb34e3bfc1b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:27.829254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750eca6669914c09bbd1aeb34e3bfc1b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:27.829874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750eca6669914c09bbd1aeb34e3bfc1b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:27.844882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:27.850632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1120/2228] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:27 2022] DELETE /identity/v3/users/750eca6669914c09bbd1aeb34e3bfc1b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:27.854941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:27.855462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:27.855462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:27.875171 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:27.876369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:27.946163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1109/2229] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:27.956143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:27.958028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.036163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883478e-a7f4-425c-83d9-4ae5911ce7fa admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.040070 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.040759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.041662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.041990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.051997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.053411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46151255-ba1d-41d3-a5d4-ee6dd2425e19 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.064311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1121/2230] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:28.414439 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.415892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.428095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b181a-4f62-430b-9ffb-bcdb2b1f3743 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.434979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.435638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.435920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.436212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] PATH_INFO: `/v3/projects/84637182526541fb9717285ed49d48a3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.437356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84637182526541fb9717285ed49d48a3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.439556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.446749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:28.490779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84637182526541fb9717285ed49d48a3', 'request_id': 'req-e535dd27-5f59-475f-a19c-b325a88f1ac1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:28.497062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84637182526541fb9717285ed49d48a3', 'request_id': 'req-e535dd27-5f59-475f-a19c-b325a88f1ac1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:28.504622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84637182526541fb9717285ed49d48a3', 'request_id': 'req-e535dd27-5f59-475f-a19c-b325a88f1ac1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:28.516074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] The token cache is being invalidate because project 84637182526541fb9717285ed49d48a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:28.516364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:28.517000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e535dd27-5f59-475f-a19c-b325a88f1ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:28.518523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1110/2231] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:28 2022] DELETE /identity/v3/projects/84637182526541fb9717285ed49d48a3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:28.523366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.523679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.523956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.543355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:28.544657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.615796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1122/2232] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:28.624297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.624605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.624878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.644420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:28.645761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.653777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1111/2233] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:28.657911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.667872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ce805-a910-4eac-a1cb-f44c49dda0d0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.672394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.672978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.673253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.673518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.674154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:15:28.674450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.676055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.688120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1123/2234] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:15:28.692400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.701463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0afb1-a4fd-4441-9926-317df8220c3c None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.706144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.706773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.707054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.707327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.707945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.709847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.736425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1112/2235] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:28.740503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.756502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca531ff8-a4a2-4f59-96ac-bf3068fbf221 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.813477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.814073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.814350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.814647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.815275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.817136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.837127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1124/2236] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:28.841752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.852357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d637c-91a1-449d-abba-6e6dacac0b1d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.861738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.862513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.862671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.862909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.863559 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.865163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.875527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1113/2237] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:28.879777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.894190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed108f-7aca-4d00-b06f-70ffd9f64ba6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.899133 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.899809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.900093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.900364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.901597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.903295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308eb6-da01-428a-b277-bf12dfc2b826 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.914445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1125/2238] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:28.919334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.929649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2f929-4672-48b7-81c5-882028f39374 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.938588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.939412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.939693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.939966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] PATH_INFO: `/v3/projects/278e12fd2a894c2c9818e14536ad647d/users/e71b4c5182c64709b65a0ba6917eb55f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.956080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278e12fd2a894c2c9818e14536ad647d, user_id=e71b4c5182c64709b65a0ba6917eb55f, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.958692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.972377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1114/2239] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:28 2022] PUT /identity/v3/projects/278e12fd2a894c2c9818e14536ad647d/users/e71b4c5182c64709b65a0ba6917eb55f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:28.984360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.984641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.984909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.003741 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:29.013101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] MFA Rules not processed for user `e71b4c5182c64709b65a0ba6917eb55f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:29.014285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.077842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1126/2240] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:29.082235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.141379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0905-55d8-4889-b0c1-cd4059af2c66 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.146212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.146855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.147142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.147419 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.148050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.149983 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.176520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1115/2241] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:29 2022] POST /identity/v3/projects => generated 373 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:29.180704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.191297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a738a43-b227-461e-adeb-5d2bfa268e0b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.245434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.245981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.246254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.246520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.247186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.249116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.288972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1127/2242] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:29 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:29.292617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.303081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37797a7-e745-4230-b20b-ecc4105fb330 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.308013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.308648 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.308946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.309207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.309851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.311469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.322055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1116/2243] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:29.326522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.336361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b56575-9cba-4d98-98c9-785731a69f5e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.341639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.342287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.342587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.342870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] PATH_INFO: `/v3/projects/7ca628b535954e93a959f5b3015f32d4/users/a953b6867f04425098e4f8748ef3aed4/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.358217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca628b535954e93a959f5b3015f32d4, user_id=a953b6867f04425098e4f8748ef3aed4, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.360874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.374871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1128/2244] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:29 2022] PUT /identity/v3/projects/7ca628b535954e93a959f5b3015f32d4/users/a953b6867f04425098e4f8748ef3aed4/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:29.378864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.385931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:29.389834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd845c3f-d367-4f50-b6f9-4eb83c34c355 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.449329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.449991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.450236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.450510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.451198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.452799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de198ad-5960-45f3-b888-d69c09122d25 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.462777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1117/2245] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:29.467110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.477562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45ba5d-0f1d-4dd7-8855-bed2271b061b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.482954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.483618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.483897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.484169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] PATH_INFO: `/v3/projects/7ca628b535954e93a959f5b3015f32d4/users/a953b6867f04425098e4f8748ef3aed4/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.486181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca628b535954e93a959f5b3015f32d4, user_id=a953b6867f04425098e4f8748ef3aed4, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.488859 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.499428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1129/2246] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:29 2022] PUT /identity/v3/projects/7ca628b535954e93a959f5b3015f32d4/users/a953b6867f04425098e4f8748ef3aed4/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:29.509417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.510309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.510735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.546322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] MFA Rules not processed for user `a953b6867f04425098e4f8748ef3aed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:29.547572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.617756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1118/2247] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:29.628754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.630546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.710367 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62c44c-cd43-4401-bbfa-8c19dc119dfc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.714111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.714748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.715032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.715307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.719602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:29.775766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.777324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d2ee75-8764-45e0-be66-5e8d9e5c4bef service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.787281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1130/2248] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:39.614826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=61 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:39.782192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=25 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:40.735098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:40.796892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c83e9f-370f-4c35-bbbe-3b2dab47bb54 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:40.801248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:40.801868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:40.802146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:40.802415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] PATH_INFO: `/v3/users/e71b4c5182c64709b65a0ba6917eb55f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:40.803577 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71b4c5182c64709b65a0ba6917eb55f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:40.805482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:40.864594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71b4c5182c64709b65a0ba6917eb55f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:40.872195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71b4c5182c64709b65a0ba6917eb55f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:40.884843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71b4c5182c64709b65a0ba6917eb55f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:40.885873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71b4c5182c64709b65a0ba6917eb55f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:40.886513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71b4c5182c64709b65a0ba6917eb55f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:40.904839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1119/2249] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:40 2022] DELETE /identity/v3/users/e71b4c5182c64709b65a0ba6917eb55f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:40.909310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:40.909765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:40.909858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:40.930355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:40.931588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:40.989629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1131/2250] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:40.999582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:41.000999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:41.057512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13ca71-1076-4d64-8b20-22d1b8eb1f21 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:41.061093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:41.061693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:41.061973 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:41.062248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:41.071409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:41.072952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:41.083054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1120/2251] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:41.357950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:41.359452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:41.375171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532fa98-4004-44c4-a079-a712eed7451f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:41.381454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:41.383379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:41.383379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:41.383379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] PATH_INFO: `/v3/projects/278e12fd2a894c2c9818e14536ad647d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:41.384730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=278e12fd2a894c2c9818e14536ad647d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:41.386525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:41.446006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278e12fd2a894c2c9818e14536ad647d', 'request_id': 'req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.455498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278e12fd2a894c2c9818e14536ad647d', 'request_id': 'req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.464165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278e12fd2a894c2c9818e14536ad647d', 'request_id': 'req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.475721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] The token cache is being invalidate because project 278e12fd2a894c2c9818e14536ad647d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:41.476009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.476642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.478369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1132/2252] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:41 2022] DELETE /identity/v3/projects/278e12fd2a894c2c9818e14536ad647d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:41.483000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:41.493328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8237888a-877b-4fd3-bcb3-528539eb3f43 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:41.513594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:41.559877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:41.560529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:41.560808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:41.561089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] PATH_INFO: `/v3/users/a953b6867f04425098e4f8748ef3aed4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:41.562251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a953b6867f04425098e4f8748ef3aed4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:41.564248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:41.612661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a953b6867f04425098e4f8748ef3aed4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.619240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a953b6867f04425098e4f8748ef3aed4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.626409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a953b6867f04425098e4f8748ef3aed4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.627068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a953b6867f04425098e4f8748ef3aed4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.627677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74b7bd8-acf6-493c-bd0d-6c6037c6659c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a953b6867f04425098e4f8748ef3aed4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:41.645133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1121/2253] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:41 2022] DELETE /identity/v3/users/a953b6867f04425098e4f8748ef3aed4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:41.936089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:41.937577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:41.952773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7debfb5-ae9f-449a-83fc-0feab14cf0f5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.008806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.009426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.009699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.010488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] PATH_INFO: `/v3/projects/7ca628b535954e93a959f5b3015f32d4` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.011619 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca628b535954e93a959f5b3015f32d4)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.013516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.043580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:42.060856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca628b535954e93a959f5b3015f32d4', 'request_id': 'req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:42.066905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca628b535954e93a959f5b3015f32d4', 'request_id': 'req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:42.074723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca628b535954e93a959f5b3015f32d4', 'request_id': 'req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:42.085483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] The token cache is being invalidate because project 7ca628b535954e93a959f5b3015f32d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:42.085765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:42.086375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c20fd8e4-2d07-4f1a-9fb8-3a5f2cabf731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:42.087881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1133/2254] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:41 2022] DELETE /identity/v3/projects/7ca628b535954e93a959f5b3015f32d4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:42.092981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.093434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.093584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.113742 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:42.115009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:42.173232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1122/2255] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:42.180382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.180667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.180939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.202295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:42.203595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:42.211583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1134/2256] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:42.215808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.225322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fb167-7fd0-42b3-a619-a81a749ed034 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.230418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.231035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.231314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.231583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.232218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:15:42.232517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.234090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.245756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1123/2257] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:15:42.249896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.259256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c5fac-30a0-4888-99e6-df6b9869bb5e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.264072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.264683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.264966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.265238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.265881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.267850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.296929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1135/2258] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:42 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:42.301049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.312243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5df20a-6502-4cf4-892a-6db4fd8bff4d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.358059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:42.392197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.392797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.393076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.393346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.393967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.395896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.419865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1124/2259] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:42 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:42.424215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.434761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc379-cea6-4a5b-9627-2fb88343b0e9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.439339 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.439947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.440230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.440501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.441137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.442889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.452812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1136/2260] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:42.456986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.467770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966dbc93-de1c-4a1a-b17a-53becd9df122 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.472595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.473191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.473472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.473741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.474384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.475987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bf09-e405-4a5e-b925-7c3bbd63e948 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.493868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1125/2261] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:42.495028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.505368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5df7c-e232-4f3f-a1ca-f7e4ce8daed3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.509986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.510668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.510955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.511234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] PATH_INFO: `/v3/projects/953eb1885e744b8497c4e41e5c7f688a/users/fd00300e201c410db75dc5b14a31150e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.526503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] RBAC: Authorizing `identity:create_grant(project_id=953eb1885e744b8497c4e41e5c7f688a, user_id=fd00300e201c410db75dc5b14a31150e, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.529162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.542977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1137/2262] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:42 2022] PUT /identity/v3/projects/953eb1885e744b8497c4e41e5c7f688a/users/fd00300e201c410db75dc5b14a31150e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:15:42.613475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.613765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.614047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.641235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] MFA Rules not processed for user `fd00300e201c410db75dc5b14a31150e`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:42.642436 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:42.710941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1126/2263] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:42.722587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:42.724369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:42.784137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce7a68-4ca4-411d-a788-6898736a037e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:42.788014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:42.788621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:42.789044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:42.789179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:42.797534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:42.798973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986d51e-7033-452e-a0ca-e98a8755b826 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:42.810368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1138/2264] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:46.365764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:46.367331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.459688 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3fe71-4e3a-42af-a8fe-22347da4093b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:46.467416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:46.468059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.468341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.468612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] PATH_INFO: `/v3/users/9f8f5fefe88e46338a57cac053799e4f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.470856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8f5fefe88e46338a57cac053799e4f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:46.473384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:46.540495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8f5fefe88e46338a57cac053799e4f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:46.541947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.543125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.543630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.543754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.550269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8f5fefe88e46338a57cac053799e4f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:46.562138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8f5fefe88e46338a57cac053799e4f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:46.562905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8f5fefe88e46338a57cac053799e4f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:46.564003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a74ad2a1-d1a5-419f-badb-caac751a2dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8f5fefe88e46338a57cac053799e4f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:46.565782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] MFA Rules not processed for user `5d1368fb915444b599b3fcf3f5586789`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:46.567267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.591610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1127/2265] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:46 2022] DELETE /identity/v3/users/9f8f5fefe88e46338a57cac053799e4f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:46.596117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.596608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.596890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.627776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:46.629045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.638758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.651080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1139/2266] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:46.707161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1128/2267] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:46.712359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:46.713979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.780875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6ae51-5b1e-4615-93ea-92a6dd78a1da None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:46.836057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:46.836861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.837215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.837551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.848677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:46.850408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e59928-7be8-4f51-85a6-8dcceed31481 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:46.861115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1140/2268] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:47.400004 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.401625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:47.421902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c9319-dda2-4aa4-95b1-b8a41ffb4301 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.427642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.428693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.428902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.429197 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] PATH_INFO: `/v3/projects/e5bd389686bd4de7a37ff25afe93c11a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.430383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5bd389686bd4de7a37ff25afe93c11a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.432473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.485876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5bd389686bd4de7a37ff25afe93c11a', 'request_id': 'req-21cdf09f-d701-41f1-943f-1d5e64ffd31d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:47.492326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5bd389686bd4de7a37ff25afe93c11a', 'request_id': 'req-21cdf09f-d701-41f1-943f-1d5e64ffd31d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:47.502395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5bd389686bd4de7a37ff25afe93c11a', 'request_id': 'req-21cdf09f-d701-41f1-943f-1d5e64ffd31d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:47.518522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] The token cache is being invalidate because project e5bd389686bd4de7a37ff25afe93c11a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:47.519025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:47.519795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-21cdf09f-d701-41f1-943f-1d5e64ffd31d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:47.522681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1129/2269] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:47 2022] DELETE /identity/v3/projects/e5bd389686bd4de7a37ff25afe93c11a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:47.526647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.526942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.527216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.552548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:47.554464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:47.635749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1141/2270] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:47.643609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.644112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.644395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.666138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:47.667388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:47.675753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1130/2271] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:47.679751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.690785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c48142-7cdf-4481-978d-3a703c729c90 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.699270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.700833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.701697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.702039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.702742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:15:47.703052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.704684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.717279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1142/2272] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:15:47.722664 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.735853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c75e59-e925-4f79-9e05-aa4bb09de168 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.741503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.742021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.742299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.742621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.743328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.745435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.780036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1131/2273] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:15:47 2022] POST /identity/v3/projects => generated 363 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:15:47.785662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.804321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93cc72-1f81-4c17-96df-4320b676e872 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.856618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:47.881150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.881840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.882128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.882399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.883073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.884972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.912366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1143/2274] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:15:47 2022] POST /identity/v3/users => generated 282 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:15:47.913644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.924593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58e38-5c25-42dc-9327-2803889f8e18 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.929561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.930209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.930493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.930791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.931466 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.933317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.939220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:47.944476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1132/2275] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:15:47.948885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.959154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750bed-b9c7-4d1f-917f-ed459cc5edf3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.963832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.964436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.964720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.964993 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:47.965638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:47.967270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a032d765-9bfd-4d71-8fa2-700b41033016 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:47.977592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1144/2276] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:15:47.982326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:47.992207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bc1dc-cc85-4afd-b74c-9f6ba92508c7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:47.996860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:47.998142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:47.998423 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:47.998737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] PATH_INFO: `/v3/projects/6ba536895e8945229b18e38b8b01b3a6/users/5a76200678a847ad82533f6c140c07b8/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:48.013982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba536895e8945229b18e38b8b01b3a6, user_id=5a76200678a847ad82533f6c140c07b8, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:48.016734 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:48.030448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1133/2277] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:15:47 2022] PUT /identity/v3/projects/6ba536895e8945229b18e38b8b01b3a6/users/5a76200678a847ad82533f6c140c07b8/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:48.041935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:48.042455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:48.042785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:48.071954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] MFA Rules not processed for user `5a76200678a847ad82533f6c140c07b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:48.073176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:48.149902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1145/2278] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:48.160640 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:48.162335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:48.217092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6dd76-8ee8-4688-b4c4-f1669a051427 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:48.220706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:48.221294 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:48.221572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:48.221844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:48.229790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:48.231224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399cb53-94ac-47fa-8e54-44c69874d9f6 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:48.240402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1134/2279] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:57.611666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:57.613127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:57.679320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af58788-f54c-4ac1-ab7e-03ddb5ceed9b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:57.684322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:57.684944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:57.685227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:57.685501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] PATH_INFO: `/v3/users/fd00300e201c410db75dc5b14a31150e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:57.686667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd00300e201c410db75dc5b14a31150e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:57.688610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:57.735421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd00300e201c410db75dc5b14a31150e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:57.741604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:57.743796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd00300e201c410db75dc5b14a31150e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:57.753000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd00300e201c410db75dc5b14a31150e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:57.754500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd00300e201c410db75dc5b14a31150e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:57.755152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd00300e201c410db75dc5b14a31150e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:57.774075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1146/2280] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:15:57 2022] DELETE /identity/v3/users/fd00300e201c410db75dc5b14a31150e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:15:57.779758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:57.780564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:57.780941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:57.806999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:57.808616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:57.872671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1135/2281] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:57.882416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:57.884029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:57.941515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df41ad9-36be-4843-a33d-435bb3e56666 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:57.945093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:57.945692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:57.945972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:57.946249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:57.955110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:57.956521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a2068-c71f-42db-a459-fd502406ed9e service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:57.965637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1147/2282] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:58.229904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:58.231383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:58.243668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ce651-dc79-4648-a392-a45d32c44fb0 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:58.248782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:58.249397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:58.249833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:58.250116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] PATH_INFO: `/v3/projects/953eb1885e744b8497c4e41e5c7f688a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:58.251277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] RBAC: Authorizing `identity:delete_project(project_id=953eb1885e744b8497c4e41e5c7f688a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:58.253218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:58.300782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953eb1885e744b8497c4e41e5c7f688a', 'request_id': 'req-77eaa467-bc1c-47bb-8907-1881c21a579a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:58.306010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953eb1885e744b8497c4e41e5c7f688a', 'request_id': 'req-77eaa467-bc1c-47bb-8907-1881c21a579a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:58.313033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953eb1885e744b8497c4e41e5c7f688a', 'request_id': 'req-77eaa467-bc1c-47bb-8907-1881c21a579a'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:58.323596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] The token cache is being invalidate because project 953eb1885e744b8497c4e41e5c7f688a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:15:58.323935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:58.324551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:15:58.326014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1136/2283] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:15:58 2022] DELETE /identity/v3/projects/953eb1885e744b8497c4e41e5c7f688a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:16:07.961671 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=35,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:08.320988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:18.929139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:18.929625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:18.929625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:18.948323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] MFA Rules not processed for user `7ebc052a3cb6400a961c1b9d9bec83d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:18.949524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:19.004241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1148/2284] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:20.276699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:20.278799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.352550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaa467-bc1c-47bb-8907-1881c21a579a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:20.358179 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:20.359076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:20.359220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:20.359499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:20.361233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.373248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:20.374841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc6b0a-0e1d-4c4f-a7d9-928b3d7dd28a service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:20.384403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1137/2285] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:20.773057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:20.774496 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.838228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdfc2f-98a3-4c51-8969-388435094fca None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:20.843224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:20.843843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:20.844119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:20.844388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] PATH_INFO: `/v3/users/db379afd44f8450c938ad721f5f4423b` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:20.845611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db379afd44f8450c938ad721f5f4423b)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:20.847528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:20.898899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db379afd44f8450c938ad721f5f4423b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:20.906470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db379afd44f8450c938ad721f5f4423b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:20.914376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db379afd44f8450c938ad721f5f4423b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:20.915053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db379afd44f8450c938ad721f5f4423b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:20.915809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db379afd44f8450c938ad721f5f4423b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:20.934337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1149/2286] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:16:20 2022] DELETE /identity/v3/users/db379afd44f8450c938ad721f5f4423b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:16:20.938931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:20.939221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:20.939499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:20.958076 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:20.959314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:21.022700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1138/2287] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:21.033573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:21.036173 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:21.080535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:21.120162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a773b9-9983-496e-921f-ecb8cdcf6ad9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:21.124299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:21.124916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.125196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.125468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.136360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:21.138347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc12851-65e5-43d3-87c9-a7db8c491d97 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:21.149833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1150/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:21.797814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:21.800090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:21.823649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2da837-da2c-4d6e-9ca1-cea48644ec67 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:21.838257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:21.838950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.839235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.839510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] PATH_INFO: `/v3/projects/575943d6b15445c1b84708e40f430196` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.840711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=575943d6b15445c1b84708e40f430196)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:21.843762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:21.906808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575943d6b15445c1b84708e40f430196', 'request_id': 'req-781b160a-830d-4aa4-9c63-9f92ea76afe1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:21.913356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575943d6b15445c1b84708e40f430196', 'request_id': 'req-781b160a-830d-4aa4-9c63-9f92ea76afe1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:21.922641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575943d6b15445c1b84708e40f430196', 'request_id': 'req-781b160a-830d-4aa4-9c63-9f92ea76afe1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:21.939546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] The token cache is being invalidate because project 575943d6b15445c1b84708e40f430196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:16:21.939897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:21.940563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-781b160a-830d-4aa4-9c63-9f92ea76afe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:21.945942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1139/2289] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:16:21 2022] DELETE /identity/v3/projects/575943d6b15445c1b84708e40f430196 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:16:21.948137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.948665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.948872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.972031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:21.973573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:22.043628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1151/2290] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:22.051641 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.051944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.052224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.082957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:22.084349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:22.094041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1140/2291] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:22.098518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.108900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ccf12-3ab4-419e-b6c8-7c163f94a192 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.117171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.117820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.118060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.119682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.119682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:16:22.119682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.120889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.134611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1152/2292] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:16:22.140442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.151084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e189ac-a598-422f-b84e-52b7165f2e34 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.160493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.160493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.160493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.160493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.160871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.164505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.170846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:22.214498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1141/2293] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:16:22 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:16:22.219072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.230932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2050921-8219-499f-9392-2483e00151ee admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.312519 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.313512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.313650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.313851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.314491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.317325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.346995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1153/2294] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:16:22 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:16:22.352292 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.362932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c57ec8-bbee-4e13-a785-854dfbba44ce admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.368219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.368986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.369272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.369550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.370212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.372366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.393230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.399215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1142/2295] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:16:22.407746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201d9b-c75c-456b-a0de-0ea168859291 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.414788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.415408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.415690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.415959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.416905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.418697 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a23cc3b-8593-4a24-a6a1-0ed61eb3c50c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.436633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1154/2296] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:16:22.440502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.453637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc64d4-d108-4175-8895-2d3ed1bb1554 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.459483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.460254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.460564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.460843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] PATH_INFO: `/v3/projects/c3124cca0402483c918d98e73bb8e0d2/users/4e49bc238f864c0e8774e8330dcf7282/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.476903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3124cca0402483c918d98e73bb8e0d2, user_id=4e49bc238f864c0e8774e8330dcf7282, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.480625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.503832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1143/2297] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:16:22 2022] PUT /identity/v3/projects/c3124cca0402483c918d98e73bb8e0d2/users/4e49bc238f864c0e8774e8330dcf7282/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:16:22.516466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.516756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.517047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.556967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] MFA Rules not processed for user `4e49bc238f864c0e8774e8330dcf7282`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:22.558445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:22.593593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:22.655962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1155/2298] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:16:22.667849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.672541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:22.746167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c391c9-7aff-42e5-83e0-a9a927b28ad2 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.750989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.751622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.751903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.752176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.761506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.763036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ea1be-4c1d-48df-b672-039f97751b13 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.772060 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1144/2299] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:22.798274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:22.881262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da3b9-3428-432a-a724-0b3621deb95f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:22.887469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:22.888241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.888607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.889237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] PATH_INFO: `/v3/users/4e49bc238f864c0e8774e8330dcf7282` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:22.890413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e49bc238f864c0e8774e8330dcf7282)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.892610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.948177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e49bc238f864c0e8774e8330dcf7282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:22.955385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e49bc238f864c0e8774e8330dcf7282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:22.963610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e49bc238f864c0e8774e8330dcf7282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:22.964477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e49bc238f864c0e8774e8330dcf7282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:22.965093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e49bc238f864c0e8774e8330dcf7282'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:22.985291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1156/2300] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:16:22 2022] DELETE /identity/v3/users/4e49bc238f864c0e8774e8330dcf7282 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:16:22.989657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:22.990117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:22.990224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.027617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:23.029578 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.094829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1145/2301] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:23.108376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:23.110683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.170942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:23.194276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbc1743-5a08-4e65-a02a-92d38472119e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:23.200175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:23.200843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:23.201128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:23.201405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.212514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:23.214254 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:23.226837 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1157/2302] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:23.711126 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:23.714232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.732137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9e717-cb46-46d6-8959-af6d83d55254 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:23.739687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:23.740967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:23.741323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:23.741777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] PATH_INFO: `/v3/projects/c3124cca0402483c918d98e73bb8e0d2` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.743216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3124cca0402483c918d98e73bb8e0d2)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:23.745814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:23.805192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:23.806606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3124cca0402483c918d98e73bb8e0d2', 'request_id': 'req-46291f48-c57d-4fd9-a6ba-01987f9950e0'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:23.813828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3124cca0402483c918d98e73bb8e0d2', 'request_id': 'req-46291f48-c57d-4fd9-a6ba-01987f9950e0'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:23.821027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3124cca0402483c918d98e73bb8e0d2', 'request_id': 'req-46291f48-c57d-4fd9-a6ba-01987f9950e0'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:23.832582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] The token cache is being invalidate because project c3124cca0402483c918d98e73bb8e0d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:16:23.832871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:23.833517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:23.835085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1146/2303] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:16:23 2022] DELETE /identity/v3/projects/c3124cca0402483c918d98e73bb8e0d2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:16:33.222258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:33.829486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:40.135125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:40.136522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:40.147512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a0cb2-e028-444c-bb5e-60068a989eeb service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:40.195357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:40.195950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:40.196231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:40.196504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:40.198228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:40.260268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:40.261682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfb2d2-ec58-44b0-b868-6509f0540976 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:40.271657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1158/2304] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:50.267476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=50 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:52.944384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:52.945787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:53.008437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46291f48-c57d-4fd9-a6ba-01987f9950e0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:53.012956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:53.013570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:53.013850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:53.014121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] PATH_INFO: `/v3/users/5a76200678a847ad82533f6c140c07b8` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:53.015753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a76200678a847ad82533f6c140c07b8)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:53.017652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:53.075495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a76200678a847ad82533f6c140c07b8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.083020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a76200678a847ad82533f6c140c07b8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.090045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a76200678a847ad82533f6c140c07b8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.090680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a76200678a847ad82533f6c140c07b8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.091280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a76200678a847ad82533f6c140c07b8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.108740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1147/2305] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:16:52 2022] DELETE /identity/v3/users/5a76200678a847ad82533f6c140c07b8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:16:53.113222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:53.113673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:53.113800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:53.133369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:53.134513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:53.191851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1159/2306] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:53.201363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:53.203282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:53.258299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9642b1-582a-4400-a45b-4ae30a25c100 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:53.261899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:53.262487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:53.262792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:53.263062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:53.272219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:53.273634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49ab51-db26-4562-98a7-d201891fe3c2 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:53.283320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1148/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:53.547871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:53.549305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:53.563982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7ecad-a913-424d-afcf-9cfde23a3043 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:53.568666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:53.569297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:53.569574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:53.569846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] PATH_INFO: `/v3/projects/6ba536895e8945229b18e38b8b01b3a6` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:53.571006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba536895e8945229b18e38b8b01b3a6)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:53.573757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:53.619715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba536895e8945229b18e38b8b01b3a6', 'request_id': 'req-9b80c778-747a-40a2-be3a-9c49cddfb379'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.625791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba536895e8945229b18e38b8b01b3a6', 'request_id': 'req-9b80c778-747a-40a2-be3a-9c49cddfb379'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.632861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba536895e8945229b18e38b8b01b3a6', 'request_id': 'req-9b80c778-747a-40a2-be3a-9c49cddfb379'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.644609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] The token cache is being invalidate because project 6ba536895e8945229b18e38b8b01b3a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:16:53.644902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.645530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:16:53.647045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1160/2308] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:16:53 2022] DELETE /identity/v3/projects/6ba536895e8945229b18e38b8b01b3a6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:17:03.278692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:03.641991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:18.441878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:18.442500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:18.442500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:18.461289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] MFA Rules not processed for user `0a30897a10d8467aa294f24954368a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:18.462499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:18.519315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1149/2309] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:28.512040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=25 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:55.149668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:55.151120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:55.206824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80c778-747a-40a2-be3a-9c49cddfb379 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:55.210441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:55.211102 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:55.211385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:55.211659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:55.213386 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:55.280744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:55.282154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:55.292780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1161/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:56.181364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:56.182705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:56.198298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f481f7-c4f1-4611-8a3f-0961205d7a3d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:56.204302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:56.205391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:56.205666 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:56.205933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:56.207604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:56.267794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:56.269382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193952e-330c-4ce1-86f6-66c09fecd48a service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:56.278807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1150/2311] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:56.330836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:56.332226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:56.386844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4d63b-31ab-4405-be8f-6eb1335b9186 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:56.390596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:56.391236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:56.391514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:56.391811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:56.394098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:56.407094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:56.408506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:56.419187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1162/2312] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:05.345226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521651b-4685-4ce5-abb6-7fd07bb513c1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:05.346405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521651b-4685-4ce5-abb6-7fd07bb513c1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:05.346405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521651b-4685-4ce5-abb6-7fd07bb513c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:05.365649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1521651b-4685-4ce5-abb6-7fd07bb513c1 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:05.366870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1521651b-4685-4ce5-abb6-7fd07bb513c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:05.419357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1151/2313] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:18:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:05.461119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:05.462632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:05.510259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:05.536376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9188d-9a3a-4b11-ada4-03700d92ae9e service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:05.542132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:05.542775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:05.543054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:05.543324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:05.554001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:05.555586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93dee29-04c0-4a42-ba10-f8c8881f2b4c service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:05.564920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1163/2314] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:18:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:18:15.412783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=53 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:15.560667 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=12 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:22.264999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.265558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.265558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.266911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1152/2315] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:18:22.277501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3677f-55fd-4020-a6b2-327d71b27006 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.278027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3677f-55fd-4020-a6b2-327d71b27006 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.278146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3677f-55fd-4020-a6b2-327d71b27006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.299362 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-35d3677f-55fd-4020-a6b2-327d71b27006 None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:22.300597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-35d3677f-55fd-4020-a6b2-327d71b27006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.311661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1164/2316] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:22.323414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:22.325118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.384244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8513f5-9761-4bb4-9934-09e64c8c4132 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:22.388006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:22.388634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.388909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.389176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.399857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:22.401286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b06fb0-d19e-46ca-9b09-ebe37421880d service glance] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:22.413187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1153/2317] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:22.507612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7697d2-5e54-45c6-9dbf-d027118684a8 None None] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.507922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7697d2-5e54-45c6-9dbf-d027118684a8 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.508199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7697d2-5e54-45c6-9dbf-d027118684a8 None None] PATH_INFO: `/` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.509577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1165/2318] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:18:22.514279 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cc4bf-878d-45b9-825e-e330b9af3885 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.514565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cc4bf-878d-45b9-825e-e330b9af3885 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.514896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40cc4bf-878d-45b9-825e-e330b9af3885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.534473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f40cc4bf-878d-45b9-825e-e330b9af3885 None None] MFA Rules not processed for user `5c5297e9a66d4124bc4888e5a92dc885`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:22.536138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f40cc4bf-878d-45b9-825e-e330b9af3885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.542903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1154/2319] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:22.554337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7697d2-5e54-45c6-9dbf-d027118684a8 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:22.565997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7697d2-5e54-45c6-9dbf-d027118684a8 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:22.569494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:22.570081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.570360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.570666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.577543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:22.581460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d5955-b3b1-49d2-8a2c-087c2ee5a5e3 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:22.589105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1166/2320] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:22.671782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ea91c-70e1-4c8a-8cd0-35129e374f5d None None] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.672069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ea91c-70e1-4c8a-8cd0-35129e374f5d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.672369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ea91c-70e1-4c8a-8cd0-35129e374f5d None None] PATH_INFO: `/` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.673815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1155/2321] 10.222.0.49 () {54 vars in 921 bytes} [Wed Aug 10 11:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:18:22.678658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.678968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.679250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.699032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] MFA Rules not processed for user `1561e7c10ad04eeda2e31c3e31cf3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:22.700237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.709961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1167/2322] 10.222.0.49 () {58 vars in 1018 bytes} [Wed Aug 10 11:18:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:22.720221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ea91c-70e1-4c8a-8cd0-35129e374f5d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:22.731107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ea91c-70e1-4c8a-8cd0-35129e374f5d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:22.734900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:22.735485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.735759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.736028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.745984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:22.747432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:22.758127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1156/2323] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:22.820455 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:22.821832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.888676 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e11a1d-285e-460c-a6ca-fa7d7e32cecb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:22.894364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:22.894996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:22.895283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:22.895556 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:22.897282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:22.908890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:22.910452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0441b8-155f-4085-8ad6-bbf73e171d67 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:22.920029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1168/2324] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:23.091271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:23.100502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547868d-dd4a-43a6-8677-3e537c7b803b service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:23.104724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:23.105621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:23.105909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:23.106189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:23.107984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:23.118440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:23.119920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee695e-f651-45cd-a8a1-c3b92fd82f92 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:23.128606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1157/2325] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:26.254115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:26.254749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:26.254749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:26.273794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] MFA Rules not processed for user `5d1368fb915444b599b3fcf3f5586789`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:26.275061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:26.281959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1169/2326] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:26.878947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:26.879389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:26.879585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:26.898545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:26.899823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:26.906958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1158/2327] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:18:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:26.919679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:26.931435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b44b84-7b0c-472c-b5ec-67138134da13 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:26.936898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:26.937817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:26.938163 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:26.938443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:26.946597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:26.948175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:26.956786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1170/2328] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:18:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:18:36.893786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=44 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:36.952893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=53 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:47.181727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:47.194680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79023929-5312-4777-b949-bea5ea5c1613 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:47.200297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:47.200966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:47.201348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:47.201798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:47.203707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:47.264237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:47.265306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:47.275390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1159/2329] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:47.953232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:47.966161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2667c5f-c25b-460c-8a0c-6de0f6be6f63 service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:47.971628 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:47.972248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:47.972525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:47.972794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:47.974530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:47.986942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:47.988495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942129bf-ef77-468a-8911-1d4d571cd25b service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:47.998812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1171/2330] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:53.536506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:53.550312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a72e39-11df-41d0-a448-c2f50523cf4b service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:53.556527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:53.557131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:53.557410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:53.557681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:53.559421 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:53.575910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:53.577549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:53.589035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1160/2331] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:55.999723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:56.000201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:56.000295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:56.025544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:56.026797 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:56.034657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1172/2332] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:18:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:56.047475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:56.058590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa84b2-aabf-4caf-a111-42f7abbf9f7c service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:56.063870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:56.064489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:56.064769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:56.065042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:56.072823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:56.074389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:56.082827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1161/2333] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:18:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:19:02.746658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:19:02.748193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:02.764674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05e4f5-9135-40da-961a-875c0ac8b729 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:19:02.770600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:19:02.771263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:02.771535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:02.771801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:02.773511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:02.786252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:19:02.787873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6284f787-4c7e-4a28-8963-6ed73249cd6c service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:19:02.797439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1173/2334] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:19:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:19:05.958726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:19:05.978217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481016-c2f4-4264-bdd7-0c41dde42989 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:19:05.984185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:19:05.985022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:05.985428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:05.985774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:05.988108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:06.002496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:19:06.004071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:19:06.015862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1162/2335] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:19:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:19:12.792742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=15 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:19:16.010788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=46 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:19:16.817973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959ea0-a79a-48ce-bfae-f59eab5fba01 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:16.818462 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959ea0-a79a-48ce-bfae-f59eab5fba01 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:16.818664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7959ea0-a79a-48ce-bfae-f59eab5fba01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:16.839099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f7959ea0-a79a-48ce-bfae-f59eab5fba01 None None] MFA Rules not processed for user `7ebc052a3cb6400a961c1b9d9bec83d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:19:16.840468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f7959ea0-a79a-48ce-bfae-f59eab5fba01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:16.911826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1174/2336] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:19:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:19:17.450291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:19:17.460269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47b4b63-ff9f-414b-a02d-afa414f296d4 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:19:17.463851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:19:17.464443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:17.464719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:17.464981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:17.466935 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:17.477761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:19:17.479395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:19:17.492979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1163/2337] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:19:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:19:18.058425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:18.058905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:18.059054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:18.078748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] MFA Rules not processed for user `0a30897a10d8467aa294f24954368a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:19:18.080137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:18.087601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1175/2338] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:19:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:19:27.488888 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:19:28.073432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=28 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:04.298153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:04.311340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31168cc9-bb51-4c30-8aa4-ef3201b0a944 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:04.317166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:04.317771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:04.318058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:04.318325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:04.320088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:04.379156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:04.380736 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a1f271-53ae-404b-8f9a-f2f472085d38 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:04.390383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1164/2339] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:14.386190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=28 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:14.511750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:14.521117 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a7414-42bc-436e-8f56-d2e696f7feae None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:14.524934 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:14.525489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:14.525768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:14.526036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:14.527842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:14.593721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:14.595193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:14.606318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1176/2340] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:14.977823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:14.978112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:14.978393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:14.997272 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] MFA Rules not processed for user `1f000354b524462bb66812138c6e4695`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:14.998480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:15.005913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1165/2341] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:18.301387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:18.303034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:18.322018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858ba245-1523-4738-b5b6-c12e734627f2 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:18.327985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:18.328987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:18.329418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:18.329896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:18.332492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:18.348034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:18.349835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:18.369269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1177/2342] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:18.418975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:18.420894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:18.451355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e35f1-e976-42a0-96c4-5934264862cb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:18.458306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:18.458861 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:18.459210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:18.459484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] PATH_INFO: `/v3/users/5d1368fb915444b599b3fcf3f5586789` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:18.460874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1368fb915444b599b3fcf3f5586789)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:18.462848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:18.528613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1368fb915444b599b3fcf3f5586789'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:18.549715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1368fb915444b599b3fcf3f5586789'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:18.568507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1368fb915444b599b3fcf3f5586789'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:18.569269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1368fb915444b599b3fcf3f5586789'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:18.570082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-557080b5-12c8-4266-82cb-ee390dedaee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1368fb915444b599b3fcf3f5586789'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:18.591749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1166/2343] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:18 2022] DELETE /identity/v3/users/5d1368fb915444b599b3fcf3f5586789 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:19.046554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.062072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7a6c2-c09d-43c3-b5ee-39dfac93f5ad service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.143869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.144632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.145069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.145348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] PATH_INFO: `/v3/projects/429a80ecb8464ef3ad67a23a06fc10f7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.146668 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429a80ecb8464ef3ad67a23a06fc10f7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.150256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.226250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429a80ecb8464ef3ad67a23a06fc10f7', 'request_id': 'req-523d6623-52c0-4928-bd7d-1620b0bf0d19'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:19.233776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429a80ecb8464ef3ad67a23a06fc10f7', 'request_id': 'req-523d6623-52c0-4928-bd7d-1620b0bf0d19'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:19.241374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429a80ecb8464ef3ad67a23a06fc10f7', 'request_id': 'req-523d6623-52c0-4928-bd7d-1620b0bf0d19'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:19.258039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] The token cache is being invalidate because project 429a80ecb8464ef3ad67a23a06fc10f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:19.258260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:19.259557 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-523d6623-52c0-4928-bd7d-1620b0bf0d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:19.266767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1178/2344] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:19 2022] DELETE /identity/v3/projects/429a80ecb8464ef3ad67a23a06fc10f7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:19.271405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.272670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.273307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.299656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:19.301739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:19.384562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1167/2345] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:19.392121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.392675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.392675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.413559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:19.415115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:19.423480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1179/2346] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:19.428318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.440027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220caf3-6661-44fe-9ab8-e3c81e9e7829 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.446422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.447164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.448585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.448865 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.449529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:19.449834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.451486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.465320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1168/2347] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:19.469736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.475806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:19.480620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b01d2-0321-4e16-a3c2-81e25ddd049a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.487753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.488847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.489377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.489851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.490724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.492648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.525020 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1180/2348] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:19 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:19.529041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.541917 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466945d-ef69-4d30-be09-fce966d244f0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.626255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:19.642862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.644344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.645108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.645722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.646715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.649304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.684514 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1169/2349] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:19 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:19.688385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.705400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7060e-a0e0-4dd2-97de-82da16c3abfc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.714802 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.715895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.716566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.716909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.717732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.720600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.735428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1181/2350] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:19.739141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.753299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf257f-cd33-424b-b1ed-c766f6c44eb5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.759363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.760264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.760872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.761309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.762120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.764236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3accbe-fbdd-4609-87da-27084ce6ecd2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.794019 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1170/2351] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:19.795199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:19.806779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e292945-a84f-4e11-978d-db80c39904c7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:19.824295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:19.825208 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.825488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.825769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] PATH_INFO: `/v3/projects/c8a18bf8987e4c30b971bde3ce4b5622/users/48717417980d41d79aade50356d1501f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.864336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a18bf8987e4c30b971bde3ce4b5622, user_id=48717417980d41d79aade50356d1501f, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:19.870830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:19.915071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1182/2352] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:19 2022] PUT /identity/v3/projects/c8a18bf8987e4c30b971bde3ce4b5622/users/48717417980d41d79aade50356d1501f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:19.926496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:19.927131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:19.927575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:19.969489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] MFA Rules not processed for user `48717417980d41d79aade50356d1501f`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:19.969889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:20.051186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1171/2353] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:20.055732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.121316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855947d6-6e4b-40f6-9891-b99964177d3e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.126006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.126703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.127012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.127319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.127962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.129838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.158064 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1183/2354] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:20 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:20.162946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.174083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d463dd-193d-464b-82b1-bc030b4e9360 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.231702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.232247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.232532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.232809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.233483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.235647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.266752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1172/2355] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:20 2022] POST /identity/v3/users => generated 300 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:20.269713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.281554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9023b9-60a9-449a-9725-63d837f2ae7a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.286766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.287380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.287660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.287932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.288613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.290205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.300948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1184/2356] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:20.305034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.315794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c312cf6-4e5e-40cb-ace8-79b10b8eb03d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.320955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.321687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.321965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.322237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] PATH_INFO: `/v3/projects/bd0470678e2a49479f7538145113f7d2/users/df92f4dfebb84008ada4e199d471d3be/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.337792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0470678e2a49479f7538145113f7d2, user_id=df92f4dfebb84008ada4e199d471d3be, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.340505 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.354595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1173/2357] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:20 2022] PUT /identity/v3/projects/bd0470678e2a49479f7538145113f7d2/users/df92f4dfebb84008ada4e199d471d3be/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:20.358376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.368739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e2d4e-179e-471d-a57c-49e055c044aa admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.425782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.426437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.426682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.426952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.427621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.429218 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fc74b-e073-4c00-afc1-c8bca75d7702 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.448204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1185/2358] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:20.452475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.468675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc0561-8174-4038-87ee-fc50081dbdbd admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.474426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.475316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.475809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.476254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] PATH_INFO: `/v3/projects/bd0470678e2a49479f7538145113f7d2/users/df92f4dfebb84008ada4e199d471d3be/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.478408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0470678e2a49479f7538145113f7d2, user_id=df92f4dfebb84008ada4e199d471d3be, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.482464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.493953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1174/2359] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:20 2022] PUT /identity/v3/projects/bd0470678e2a49479f7538145113f7d2/users/df92f4dfebb84008ada4e199d471d3be/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:20.504511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.504804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.505085 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.542775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] MFA Rules not processed for user `df92f4dfebb84008ada4e199d471d3be`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:20.544083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:20.554251 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:20.668828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1186/2360] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:20.677110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.679127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:20.703554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:20.757212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a010c-3831-454a-b6a1-4cc0c3b4dffb admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.761361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c22593-d436-477e-b373-80796574925a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.761982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c22593-d436-477e-b373-80796574925a service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.762265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c22593-d436-477e-b373-80796574925a service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.762538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c22593-d436-477e-b373-80796574925a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.772679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c22593-d436-477e-b373-80796574925a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.774145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c22593-d436-477e-b373-80796574925a service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.784185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1175/2361] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:20:20.815580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:20.888917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f24e19-d847-4a62-b7f4-27cb70261e9e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:20.893933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:20.894690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:20.895146 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:20.895478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] PATH_INFO: `/v3/users/48717417980d41d79aade50356d1501f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:20.897019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=48717417980d41d79aade50356d1501f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:20.899050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:20.972446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48717417980d41d79aade50356d1501f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:20.981692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48717417980d41d79aade50356d1501f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.011807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48717417980d41d79aade50356d1501f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.011807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48717417980d41d79aade50356d1501f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.011807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48717417980d41d79aade50356d1501f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.042657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1187/2362] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:20 2022] DELETE /identity/v3/users/48717417980d41d79aade50356d1501f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:21.050388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:21.050985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:21.051499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:21.084766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:21.085746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:21.184859 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1176/2363] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:21.202317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:21.204427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:21.236838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:21.291887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f25b7a-251a-4b03-a452-fe95a3a9a09b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:21.297963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:21.298701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:21.299034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:21.300018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:21.310815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:21.312309 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:21.323953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1188/2364] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:21.775509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:21.777083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:21.791487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5316c-c87a-4cc6-8de6-7e266d30af26 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:21.797135 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:21.797784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:21.798066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:21.798341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] PATH_INFO: `/v3/projects/c8a18bf8987e4c30b971bde3ce4b5622` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:21.807368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a18bf8987e4c30b971bde3ce4b5622)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:21.809358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:21.869078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a18bf8987e4c30b971bde3ce4b5622', 'request_id': 'req-5ca37bcf-1471-4e9b-b046-b95bcb54f921'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.875591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a18bf8987e4c30b971bde3ce4b5622', 'request_id': 'req-5ca37bcf-1471-4e9b-b046-b95bcb54f921'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.884141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a18bf8987e4c30b971bde3ce4b5622', 'request_id': 'req-5ca37bcf-1471-4e9b-b046-b95bcb54f921'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.896295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] The token cache is being invalidate because project c8a18bf8987e4c30b971bde3ce4b5622 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:21.896587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.897234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:21.898746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1177/2365] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:21 2022] DELETE /identity/v3/projects/c8a18bf8987e4c30b971bde3ce4b5622 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:21.902937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:21.913600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a164f1-5e93-4991-98df-7876ec6e0f64 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:21.969932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:21.970564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:21.971389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:21.971665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] PATH_INFO: `/v3/users/df92f4dfebb84008ada4e199d471d3be` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:21.972799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df92f4dfebb84008ada4e199d471d3be)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:21.974728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:22.024185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df92f4dfebb84008ada4e199d471d3be'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.031720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df92f4dfebb84008ada4e199d471d3be'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.039736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df92f4dfebb84008ada4e199d471d3be'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.040412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df92f4dfebb84008ada4e199d471d3be'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.041015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ef2870aa-d5d8-4a14-a106-395dfee75b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df92f4dfebb84008ada4e199d471d3be'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.059276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1189/2366] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:21 2022] DELETE /identity/v3/users/df92f4dfebb84008ada4e199d471d3be => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:22.480095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:22.482118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:22.516772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca37bcf-1471-4e9b-b046-b95bcb54f921 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:22.600394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:22.631392 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:22.632415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:22.632824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:22.633201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] PATH_INFO: `/v3/projects/bd0470678e2a49479f7538145113f7d2` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:22.634625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0470678e2a49479f7538145113f7d2)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:22.636894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:22.696625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0470678e2a49479f7538145113f7d2', 'request_id': 'req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.702530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0470678e2a49479f7538145113f7d2', 'request_id': 'req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.711886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0470678e2a49479f7538145113f7d2', 'request_id': 'req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.725620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] The token cache is being invalidate because project bd0470678e2a49479f7538145113f7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:22.725910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.726607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-88a1b7f4-5e0a-4665-8eca-5946cbca12eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:22.728121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1178/2367] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:22 2022] DELETE /identity/v3/projects/bd0470678e2a49479f7538145113f7d2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:22.742185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:22.742185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:22.742185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:22.755349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:22.756669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:22.800957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:22.822943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1190/2368] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:22.831413 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:22.831934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:22.831934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:22.854305 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:22.855999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:22.866043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1179/2369] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:22.870136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:22.880372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb340118-3e18-4756-b73b-fb97a0d50833 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:22.885172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:22.885769 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:22.886048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:22.886320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:22.887000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:22.887357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:22.888910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:22.904304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1191/2370] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:22.905422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:22.915016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37728d48-7210-43f7-8d37-0d20397e87e1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:22.919713 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:22.920242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:22.920521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:22.920789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:22.921462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:22.923390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:22.954367 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1180/2371] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:22 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:22.960452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:22.973694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76ed1f-3b88-4149-9e5e-a9215f6c72f5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.032305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.032932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.033241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.033658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.034299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.036271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.056975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1192/2372] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:22 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:23.061200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.073379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685144c6-9f81-46ea-994a-abce10282aba admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.077770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.078386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.078698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.078972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.079628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.081796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.092101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1181/2373] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:23.096910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.107510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b2ffbe-6955-4921-9776-7b5d7d691a43 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.112865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.113470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.113751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.114024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.114756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.116341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36187201-61f3-418f-bed1-35ad08d45135 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.127718 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1193/2374] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:23.131989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.142383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24d8d9-bfeb-4a71-9c94-3e30f7bee5e1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.147723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.148431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.148775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.149113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] PATH_INFO: `/v3/projects/f76ce1b1ce39480299e6fc20f27cd1c0/users/eb093f6a179e475483d95c2a2b3cdc77/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.164217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76ce1b1ce39480299e6fc20f27cd1c0, user_id=eb093f6a179e475483d95c2a2b3cdc77, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.167184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.179800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1182/2375] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:23 2022] PUT /identity/v3/projects/f76ce1b1ce39480299e6fc20f27cd1c0/users/eb093f6a179e475483d95c2a2b3cdc77/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:23.189892 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75de8-75c9-4e7e-9d16-f56858de6cce None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.190175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75de8-75c9-4e7e-9d16-f56858de6cce None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.190447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b75de8-75c9-4e7e-9d16-f56858de6cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.217915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-10b75de8-75c9-4e7e-9d16-f56858de6cce None None] MFA Rules not processed for user `eb093f6a179e475483d95c2a2b3cdc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:23.219150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-10b75de8-75c9-4e7e-9d16-f56858de6cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.286978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1194/2376] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:23.291608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.336151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.336510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.336857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.354760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7223599-2229-4927-9ac7-61a4b5f33750 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.356291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] MFA Rules not processed for user `951cabf887f14896b0c4402b686be697`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:23.357663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.359432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.360017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.360300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.360574 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.361196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.363749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.374879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:23.397150 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1183/2377] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:23 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:23.397360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:23.398340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.419775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1195/2378] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:20:23.461651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b48a390-c575-4e98-8211-7f508e43a49c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.466406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.467049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.467333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.467608 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.468403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.470130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.493457 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1184/2379] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:23 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:23.497537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.508561 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c796e-c514-42a9-873f-436931f2637c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.513165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.513762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.514063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.514316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.515002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.516626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.529798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1196/2380] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:23.530949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.541310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09914825-9b27-4470-a55a-9d73eb225e29 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.545944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.546614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.546897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.547172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] PATH_INFO: `/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae/users/24c8ff66c99d403dbd108f2c9d9a8c08/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.562723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb112bcaabd4dd7be628efa09bcd0ae, user_id=24c8ff66c99d403dbd108f2c9d9a8c08, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.565545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.579595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1185/2381] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:23 2022] PUT /identity/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae/users/24c8ff66c99d403dbd108f2c9d9a8c08/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:23.584324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.594257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50597009-df38-455a-805e-be2aa637dfbd admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.650651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.651263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.651648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.651874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.652523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.654115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe7da0-bfc0-4da4-ae67-a4dc8b75e4bc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.664658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1197/2382] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:23.668987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.680798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a93ca-3bb4-4df2-aa65-4e8d0e8cb906 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.685677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.686340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.686660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.686944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] PATH_INFO: `/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae/users/24c8ff66c99d403dbd108f2c9d9a8c08/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.688881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb112bcaabd4dd7be628efa09bcd0ae, user_id=24c8ff66c99d403dbd108f2c9d9a8c08, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.691980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.702342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1186/2383] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:23 2022] PUT /identity/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae/users/24c8ff66c99d403dbd108f2c9d9a8c08/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:23.713401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.713689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.713965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.740775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.742901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.746772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] MFA Rules not processed for user `24c8ff66c99d403dbd108f2c9d9a8c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:23.748076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.810512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40deec21-7a76-4f57-8bd5-70cb3c58cec7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.816176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.817050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.817330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.817605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.819464 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.829892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.831895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.836237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1198/2384] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:23.840806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1187/2385] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:23.847688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:23.850630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:23.908569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405731a-ca2e-4783-8251-9f6436603446 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:23.912304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c86639-e513-451c-a967-174868321334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:23.913009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86639-e513-451c-a967-174868321334 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:23.913439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86639-e513-451c-a967-174868321334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:23.913780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86639-e513-451c-a967-174868321334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:23.923369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86639-e513-451c-a967-174868321334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:23.925547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86639-e513-451c-a967-174868321334 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:23.929683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:23.936560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1199/2386] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:23 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:24.004244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:24.053662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:24.070563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bcbcc6-3cab-4f81-931f-1d682340db90 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:24.075302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:24.075931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:24.076210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:24.076482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] PATH_INFO: `/v3/users/eb093f6a179e475483d95c2a2b3cdc77` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:24.077651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb093f6a179e475483d95c2a2b3cdc77)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:24.079616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:24.125643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb093f6a179e475483d95c2a2b3cdc77'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.133379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb093f6a179e475483d95c2a2b3cdc77'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.140342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb093f6a179e475483d95c2a2b3cdc77'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.140988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb093f6a179e475483d95c2a2b3cdc77'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.141637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb093f6a179e475483d95c2a2b3cdc77'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.161255 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1188/2387] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:24 2022] DELETE /identity/v3/users/eb093f6a179e475483d95c2a2b3cdc77 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:24.165771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:24.166507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:24.166970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:24.189127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:24.190896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:24.262608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1200/2388] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:24.275374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:24.276816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:24.358229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9885a59-9217-44ad-a46d-3c5b4a2246f0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:24.363857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:24.364565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:24.364940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:24.365307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:24.375300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:24.377458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:24.388554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1189/2389] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:20:24.679900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:24.681515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:24.700746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be0eae-b649-4c64-96d9-81ab69e3308e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:24.709235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:24.709966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:24.710252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:24.710532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] PATH_INFO: `/v3/projects/f76ce1b1ce39480299e6fc20f27cd1c0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:24.711713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76ce1b1ce39480299e6fc20f27cd1c0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:24.713614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:24.763319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76ce1b1ce39480299e6fc20f27cd1c0', 'request_id': 'req-83e125aa-b2e5-4f8d-84e4-12f87291d151'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.769356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76ce1b1ce39480299e6fc20f27cd1c0', 'request_id': 'req-83e125aa-b2e5-4f8d-84e4-12f87291d151'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.776722 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76ce1b1ce39480299e6fc20f27cd1c0', 'request_id': 'req-83e125aa-b2e5-4f8d-84e4-12f87291d151'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.788618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] The token cache is being invalidate because project f76ce1b1ce39480299e6fc20f27cd1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:24.788911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.789549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.791078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1201/2390] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:24 2022] DELETE /identity/v3/projects/f76ce1b1ce39480299e6fc20f27cd1c0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:24.794992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:24.805208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3e3f9-f3c2-4a9a-ab46-4aa467dee7f8 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:24.870862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:24.872230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:24.872516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:24.872794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] PATH_INFO: `/v3/users/24c8ff66c99d403dbd108f2c9d9a8c08` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:24.874603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c8ff66c99d403dbd108f2c9d9a8c08)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:24.876597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:24.888700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:24.928484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c8ff66c99d403dbd108f2c9d9a8c08'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.935321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c8ff66c99d403dbd108f2c9d9a8c08'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.944245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c8ff66c99d403dbd108f2c9d9a8c08'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.944952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c8ff66c99d403dbd108f2c9d9a8c08'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.945570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ef27262c-1f2d-4bc9-8c51-4acafdae5237 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c8ff66c99d403dbd108f2c9d9a8c08'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:24.963563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1190/2391] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:24 2022] DELETE /identity/v3/users/24c8ff66c99d403dbd108f2c9d9a8c08 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:25.325707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.327235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:25.342753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e125aa-b2e5-4f8d-84e4-12f87291d151 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.435895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.436524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.436801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.437071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] PATH_INFO: `/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.438192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb112bcaabd4dd7be628efa09bcd0ae)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.440905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.469623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:25.497778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb112bcaabd4dd7be628efa09bcd0ae', 'request_id': 'req-98e3171e-d3e4-4af5-be09-3b39073e3c7c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:25.503369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb112bcaabd4dd7be628efa09bcd0ae', 'request_id': 'req-98e3171e-d3e4-4af5-be09-3b39073e3c7c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:25.510830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb112bcaabd4dd7be628efa09bcd0ae', 'request_id': 'req-98e3171e-d3e4-4af5-be09-3b39073e3c7c'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:25.521851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] The token cache is being invalidate because project 2fb112bcaabd4dd7be628efa09bcd0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:25.522141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:25.522812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-98e3171e-d3e4-4af5-be09-3b39073e3c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:25.524285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1202/2392] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:25 2022] DELETE /identity/v3/projects/2fb112bcaabd4dd7be628efa09bcd0ae => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:25.529165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.529759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.530042 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.550298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:25.551518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:25.607529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1191/2393] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:25.616820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.617186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.617538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.637804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:25.639042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:25.646869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1203/2394] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:25.650977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.661175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7da6e-a1d2-439f-beff-b0ad43e00565 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.666158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.666971 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.667263 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.667550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.668546 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:25.668766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.671043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.683867 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1192/2395] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:25.688171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.697483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb528e5-ae46-489e-ae02-7d5068a13b91 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.702113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.702735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.703021 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.703293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.703925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.705905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.732670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1204/2396] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:25 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:25.737550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.747692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c13e55-a4e6-48b5-a916-26d193047edd admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.805990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.806624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.806907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.807178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.807813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.809727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.835726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1193/2397] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:25 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:25.836809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.847781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab65f5b-e8d7-46aa-bb57-0a1c259b2f62 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.857803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.858417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.858739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.859012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.859788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.861540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.873340 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1205/2398] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:20:25.877283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.884114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:25.888307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae708ba-78b5-4b2a-9bb2-fcff1f6f0d91 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.896878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.897744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.898194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.898665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.899376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.900970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04189fc-3a02-4bd1-be5b-0004560cfebf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.913422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1194/2399] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:25.917734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:25.927986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24a9f8-5f0f-4a08-b575-9d625bcec059 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:25.933279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:25.933958 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.934243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.934509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] PATH_INFO: `/v3/projects/979f87de9d2c47e08f4af9f5270ae6ff/users/6d2e7fe937244af190141d1a46745de7/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:25.949470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=979f87de9d2c47e08f4af9f5270ae6ff, user_id=6d2e7fe937244af190141d1a46745de7, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:25.952132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:25.965535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1206/2400] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:25 2022] PUT /identity/v3/projects/979f87de9d2c47e08f4af9f5270ae6ff/users/6d2e7fe937244af190141d1a46745de7/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:25.976038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:25.976327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:25.976626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.004901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] MFA Rules not processed for user `6d2e7fe937244af190141d1a46745de7`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:26.006116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:26.075048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1195/2401] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:20:26.079382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.143723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a13c40-0055-4dba-b018-4f54375b6c68 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.149217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.149949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.150292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.150651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.151357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.153793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.190091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1207/2402] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:26 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:26.191845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.203257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923c747-7bf1-4159-bb6a-30065a2d2ec2 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.279524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.280258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.280647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.280994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.281766 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.285525 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.317333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1196/2403] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:26 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:26.317879 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.329290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d2805-a5cd-48b4-a2b7-31bc699d82b7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.335643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.336263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.336569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.336842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.339196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.341097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.356431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1208/2404] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:20:26.360356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.370393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0263c86b-14cb-42be-8c2a-ffc8751a5d34 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.375306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.375956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.376242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.376511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] PATH_INFO: `/v3/projects/06c344289f64414fade13a155a7c66a1/users/85eb46f62203483c94c767a948b53ff7/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.396602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c344289f64414fade13a155a7c66a1, user_id=85eb46f62203483c94c767a948b53ff7, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.399378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.421531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1197/2405] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:26 2022] PUT /identity/v3/projects/06c344289f64414fade13a155a7c66a1/users/85eb46f62203483c94c767a948b53ff7/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:26.426065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.440364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99806a5d-d2d4-4bd1-9155-e98dc6edc318 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.463984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:26.513750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.515604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.515604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.515604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.516515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.518608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e4507b-17a4-4173-abfb-18f934597eb1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.534830 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1209/2406] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:26.537157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.549454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0016a-f4ec-4a59-b422-69acf1121e88 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.556303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.556977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.557261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.557537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] PATH_INFO: `/v3/projects/06c344289f64414fade13a155a7c66a1/users/85eb46f62203483c94c767a948b53ff7/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.560116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c344289f64414fade13a155a7c66a1, user_id=85eb46f62203483c94c767a948b53ff7, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.563068 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.586991 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1198/2407] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:26 2022] PUT /identity/v3/projects/06c344289f64414fade13a155a7c66a1/users/85eb46f62203483c94c767a948b53ff7/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:26.588956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.589970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.589970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.620402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] MFA Rules not processed for user `85eb46f62203483c94c767a948b53ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:26.621644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:26.707509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1210/2408] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:26.718914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:26.720965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:26.771269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:26.791407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c66c7-4868-4bd1-a08d-793366f6e34c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:26.796619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:26.797282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:26.797565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:26.797872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:26.811065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:26.812915 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af73278e-da37-4d6d-92ec-69925c9adb45 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:26.826852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1199/2409] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:27.585087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:27.644710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b583ca11-0662-4f82-8c69-4bef23939a93 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:27.649238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:27.649854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:27.650137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:27.650411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] PATH_INFO: `/v3/users/6d2e7fe937244af190141d1a46745de7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:27.651563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2e7fe937244af190141d1a46745de7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:27.653457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:27.699012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:27.700648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2e7fe937244af190141d1a46745de7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:27.708665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2e7fe937244af190141d1a46745de7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:27.716235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2e7fe937244af190141d1a46745de7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:27.716836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2e7fe937244af190141d1a46745de7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:27.717456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2e7fe937244af190141d1a46745de7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:27.735258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1211/2410] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:27 2022] DELETE /identity/v3/users/6d2e7fe937244af190141d1a46745de7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:27.739706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:27.740216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:27.740316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:27.761847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:27.763160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:27.828008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1200/2411] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:27.837964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:27.839503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:27.914381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669f59-2496-4826-9d7a-2a88ef3cbe0c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:27.922289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:27.922933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:27.923214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:27.923487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:27.934673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:27.936219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:27.946813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1212/2412] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:28.214081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:28.215587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:28.228639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca57a59-3f9c-4fa0-ba44-b56f0e0b4d10 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:28.235857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:28.236509 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:28.237299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:28.237569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] PATH_INFO: `/v3/projects/979f87de9d2c47e08f4af9f5270ae6ff` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:28.239427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=979f87de9d2c47e08f4af9f5270ae6ff)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:28.241346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:28.288535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979f87de9d2c47e08f4af9f5270ae6ff', 'request_id': 'req-adecde70-f1de-4b30-b256-8df62d5e93fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.293946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979f87de9d2c47e08f4af9f5270ae6ff', 'request_id': 'req-adecde70-f1de-4b30-b256-8df62d5e93fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.301508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979f87de9d2c47e08f4af9f5270ae6ff', 'request_id': 'req-adecde70-f1de-4b30-b256-8df62d5e93fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.312566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] The token cache is being invalidate because project 979f87de9d2c47e08f4af9f5270ae6ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:28.312855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.313477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.315598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1201/2413] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:28 2022] DELETE /identity/v3/projects/979f87de9d2c47e08f4af9f5270ae6ff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:28.319911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:28.329258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d04b-9a13-4736-b121-32a6ea004780 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:28.384764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:28.385381 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:28.385660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:28.385926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] PATH_INFO: `/v3/users/85eb46f62203483c94c767a948b53ff7` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:28.387069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85eb46f62203483c94c767a948b53ff7)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:28.388955 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:28.434035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85eb46f62203483c94c767a948b53ff7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.441903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85eb46f62203483c94c767a948b53ff7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.448658 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85eb46f62203483c94c767a948b53ff7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.449277 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85eb46f62203483c94c767a948b53ff7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.449903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-614e7896-9935-4329-bbe6-295d6c48ad31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85eb46f62203483c94c767a948b53ff7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.454342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:28.469013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1213/2414] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:28 2022] DELETE /identity/v3/users/85eb46f62203483c94c767a948b53ff7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:28.767504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:28.769672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:28.788447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adecde70-f1de-4b30-b256-8df62d5e93fe admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:28.870282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:28.870963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:28.871275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:28.871534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] PATH_INFO: `/v3/projects/06c344289f64414fade13a155a7c66a1` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:28.872772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c344289f64414fade13a155a7c66a1)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:28.874770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:28.878095 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:28.933852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c344289f64414fade13a155a7c66a1', 'request_id': 'req-9bcc413f-ed7d-42c6-8504-d026cf247e29'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.940974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c344289f64414fade13a155a7c66a1', 'request_id': 'req-9bcc413f-ed7d-42c6-8504-d026cf247e29'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.952348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c344289f64414fade13a155a7c66a1', 'request_id': 'req-9bcc413f-ed7d-42c6-8504-d026cf247e29'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.967384 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] The token cache is being invalidate because project 06c344289f64414fade13a155a7c66a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:28.967676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.968482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9bcc413f-ed7d-42c6-8504-d026cf247e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:28.970360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1202/2415] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:28 2022] DELETE /identity/v3/projects/06c344289f64414fade13a155a7c66a1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:28.975752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:28.976310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:28.976649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:28.999026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:29.000581 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.061738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1214/2416] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:29.071474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.071675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.071932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.091121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:29.092327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.100230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1203/2417] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:29.104466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.114695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41e36f-d8cb-43b8-87d1-bcbb6d095b60 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.119400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.120094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.120376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.120735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.121450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:29.121818 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.123503 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.139428 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1215/2418] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:29.140660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.150280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee738f-57bb-4927-b5b9-f4f9f89c1ddc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.155022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.155623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.155904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.156177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.156809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.158735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.187246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1204/2419] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:29 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:29.191113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.201268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99643-ace2-49b6-83b6-30c9cc30f41a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.286212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.286886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.287139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.287410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.288035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.289936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.316014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1216/2420] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:29 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:29.320583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.332053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f6e52-551d-44f3-a66f-791f11cde824 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.337663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.338364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.338712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.338992 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.339665 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.341256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.354707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1205/2421] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:20:29.356078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.366351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512bfc2d-02a1-43da-8fd2-8cb6ae821411 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.371453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.372060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.372341 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.372618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.373271 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.375198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08cb919-566c-4b4b-a39e-9b79c194f802 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.387298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1217/2422] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:29.391706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.402599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7706-3306-47dd-97a7-cd7902a248ba admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.407404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.408051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.408333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.409144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] PATH_INFO: `/v3/projects/4075b25a8aa14f929d70b8ced451f478/users/08c7de9660754d5ba8fe47436e207a7d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.423817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4075b25a8aa14f929d70b8ced451f478, user_id=08c7de9660754d5ba8fe47436e207a7d, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.426616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.442972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1206/2423] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:29 2022] PUT /identity/v3/projects/4075b25a8aa14f929d70b8ced451f478/users/08c7de9660754d5ba8fe47436e207a7d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:29.448073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cc2c7-0469-4e77-a92a-634fb777ffd9 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.448363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cc2c7-0469-4e77-a92a-634fb777ffd9 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.448690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cc2c7-0469-4e77-a92a-634fb777ffd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.467224 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0d9cc2c7-0469-4e77-a92a-634fb777ffd9 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:29.468413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0d9cc2c7-0469-4e77-a92a-634fb777ffd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.514487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:29.530614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1218/2424] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:29.540873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.542564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.600511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad58c89-f5dd-427d-8158-f7e2fc0eebe3 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.604537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.605327 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.605604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.605878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.615495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.616931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.626418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1207/2425] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:30.552938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.553472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.553472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.590828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] MFA Rules not processed for user `08c7de9660754d5ba8fe47436e207a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:30.592138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:30.666093 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1219/2426] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:30.671014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.681928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e2ddc-3f2b-46de-beb5-f0778b8bee46 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.686684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.687361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.687711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.688049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.688754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.691411 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:30.722387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1208/2427] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:30 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:30.723425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.733875 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1234fc-df53-472c-b383-0eece953f8cb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.791694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.792326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.792616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.792890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.793525 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.795461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:30.817196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1220/2428] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:30 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:30.821522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.833447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2d83f-29f6-4ed8-bee0-95954ae484e9 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.838045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.838675 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.839007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.839283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.839919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.841567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:30.843724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:30.857043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1209/2429] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:30.861229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.875002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811294b0-10c0-4bfb-9bde-7616e6325699 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.880677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.881332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.881612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.881882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] PATH_INFO: `/v3/projects/08c34154e71149eea92c630f214d4176/users/b91ec8137651488e81360a5aaa66b2e8/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.898089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c34154e71149eea92c630f214d4176, user_id=b91ec8137651488e81360a5aaa66b2e8, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.900782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:30.913708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1221/2430] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:30 2022] PUT /identity/v3/projects/08c34154e71149eea92c630f214d4176/users/b91ec8137651488e81360a5aaa66b2e8/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:30.918024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.928249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c89c8-54c4-45bf-b104-a230de326b69 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.986746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.987302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.987611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.987884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.988557 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.990159 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6f3c8-9132-4f03-a2df-ff0b6e1d7fc4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:31.006714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1210/2431] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:31.010942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:31.022122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae03276-ff94-42f1-98ef-be2d971d92ec admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:31.026913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:31.027594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:31.027947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:31.028231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] PATH_INFO: `/v3/projects/08c34154e71149eea92c630f214d4176/users/b91ec8137651488e81360a5aaa66b2e8/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:31.030206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c34154e71149eea92c630f214d4176, user_id=b91ec8137651488e81360a5aaa66b2e8, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:31.032878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:31.044091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1222/2432] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:31 2022] PUT /identity/v3/projects/08c34154e71149eea92c630f214d4176/users/b91ec8137651488e81360a5aaa66b2e8/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:31.719082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:31.719900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:31.720181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:31.760507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] MFA Rules not processed for user `b91ec8137651488e81360a5aaa66b2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:31.761226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:31.857494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1211/2433] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:31.869189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:31.871024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:31.918962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:31.953008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f8963-b3f2-4840-994b-c7d46342cd5b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:31.957744 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:31.958507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:31.958913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:31.959285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:32.022599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:32.024054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:32.034441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1223/2434] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:33.402815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:33.404237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:33.416579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05576f1f-75dc-4d9d-9ae6-01bd4d6cc057 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:33.420122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:33.420716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:33.421018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:33.421296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:33.430055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:33.431504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:33.440727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1212/2435] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:42.029158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=33 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:43.436618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=70 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:45.823450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:45.825003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:45.908375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3b774-f152-4800-8da0-40c03ddf2aaa service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:45.914082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:45.914717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:45.914996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:45.915269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:45.917004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:45.928545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:45.930192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:45.940252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1224/2436] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:46.404020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:46.464600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c794f-7da9-4dec-a6c2-54a58938ab40 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:46.469054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:46.469667 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:46.469946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:46.470220 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] PATH_INFO: `/v3/users/08c7de9660754d5ba8fe47436e207a7d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:46.471393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c7de9660754d5ba8fe47436e207a7d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:46.473290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:46.517452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c7de9660754d5ba8fe47436e207a7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.524671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c7de9660754d5ba8fe47436e207a7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.530987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c7de9660754d5ba8fe47436e207a7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.531757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c7de9660754d5ba8fe47436e207a7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.532362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c7de9660754d5ba8fe47436e207a7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.548969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1213/2437] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:46 2022] DELETE /identity/v3/users/08c7de9660754d5ba8fe47436e207a7d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:46.721916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:46.732449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab80fdb-9609-439e-aa6f-abe604b4d752 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:46.790110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:46.790737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:46.791023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:46.791307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] PATH_INFO: `/v3/projects/4075b25a8aa14f929d70b8ced451f478` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:46.792575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4075b25a8aa14f929d70b8ced451f478)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:46.794471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:46.840009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4075b25a8aa14f929d70b8ced451f478', 'request_id': 'req-4ea89d84-c1e7-4ea9-b8de-761f26a33446'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.845375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4075b25a8aa14f929d70b8ced451f478', 'request_id': 'req-4ea89d84-c1e7-4ea9-b8de-761f26a33446'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.853454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4075b25a8aa14f929d70b8ced451f478', 'request_id': 'req-4ea89d84-c1e7-4ea9-b8de-761f26a33446'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.865437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] The token cache is being invalidate because project 4075b25a8aa14f929d70b8ced451f478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:46.865715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.866375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.867834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1225/2438] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:46 2022] DELETE /identity/v3/projects/4075b25a8aa14f929d70b8ced451f478 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:46.871972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:46.873370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:46.886657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b53591-83ff-43cc-b6eb-ba140561d82f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:46.941416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:46.942036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:46.942588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:46.942881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] PATH_INFO: `/v3/users/b91ec8137651488e81360a5aaa66b2e8` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:46.943979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b91ec8137651488e81360a5aaa66b2e8)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:46.945878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:46.966459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:46.992357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b91ec8137651488e81360a5aaa66b2e8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:46.999006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b91ec8137651488e81360a5aaa66b2e8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.005902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b91ec8137651488e81360a5aaa66b2e8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.006541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b91ec8137651488e81360a5aaa66b2e8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.007168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-74a3443c-9d5b-4b2b-8616-0b5843486395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b91ec8137651488e81360a5aaa66b2e8'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.023694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1214/2439] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:46 2022] DELETE /identity/v3/users/b91ec8137651488e81360a5aaa66b2e8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:47.200499 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.202026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.214665 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea89d84-c1e7-4ea9-b8de-761f26a33446 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.232588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:47.273814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.274440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.274757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.275031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] PATH_INFO: `/v3/projects/08c34154e71149eea92c630f214d4176` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.276138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c34154e71149eea92c630f214d4176)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.278025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.321652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c34154e71149eea92c630f214d4176', 'request_id': 'req-d9bb581b-6871-46c6-a94f-eb911f79759b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.326935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c34154e71149eea92c630f214d4176', 'request_id': 'req-d9bb581b-6871-46c6-a94f-eb911f79759b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.335192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c34154e71149eea92c630f214d4176', 'request_id': 'req-d9bb581b-6871-46c6-a94f-eb911f79759b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.346204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] The token cache is being invalidate because project 08c34154e71149eea92c630f214d4176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:47.347131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.347758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d9bb581b-6871-46c6-a94f-eb911f79759b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:47.349219 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1226/2440] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:47 2022] DELETE /identity/v3/projects/08c34154e71149eea92c630f214d4176 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:47.353997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.354281 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.354554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.374044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:47.375258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.432851 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1215/2441] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:47.441196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.441483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.441757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.461112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:47.462301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.470176 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1227/2442] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:47.474231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.484119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3441f3-71db-4eae-9c98-f6dc6d2f9b28 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.489121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.489705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.489980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.490248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.490946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:47.491247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.492825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.503980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1216/2443] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:47.508057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.517147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6c9bd-6c0a-4849-a478-adb78a2014fc None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.522240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.522855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.523136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.523408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.524038 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.525911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.553316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1228/2444] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:47.557127 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.567448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a3839-18ae-441c-bca0-ba48b0ba09fe admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.622563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.623124 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.623402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.623676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.624293 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.626211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.647428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1217/2445] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:47.651477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.661659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079975bf-7cf1-4e8d-b746-ae3e8d58c4c3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.666347 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.667001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.667258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.667576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.668281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.670029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.679795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1229/2446] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:47.683566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.694195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f43255-3dd6-445f-a608-58d83568e07e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.698925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.699512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.699794 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.700072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.700708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.702329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad533829-5693-4d83-9290-cc9d1a30454b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.711994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1218/2447] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:47.721019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.731392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1714a39-15d6-41b3-85b5-e21d3ad147b1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.736029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.736673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.736954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.737827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] PATH_INFO: `/v3/projects/5405ae53c5c34c0fbd7a91d8ffc0ef3d/users/25ed02f0e7654f0493a77ecea1285555/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.752125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5405ae53c5c34c0fbd7a91d8ffc0ef3d, user_id=25ed02f0e7654f0493a77ecea1285555, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.754782 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.768113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1230/2448] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:47 2022] PUT /identity/v3/projects/5405ae53c5c34c0fbd7a91d8ffc0ef3d/users/25ed02f0e7654f0493a77ecea1285555/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:47.778416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.778750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.779030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.787460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:47.807518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] MFA Rules not processed for user `25ed02f0e7654f0493a77ecea1285555`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:47.808745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.873056 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1219/2449] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:47.883951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.885728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.936900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:47.943908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d184e-0ced-4ac0-bad7-1609496f00d0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.948249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.949031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.949318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.949592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.958353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.959852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8883fdb-5499-4606-93e1-02a451bcfeaa service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.968474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1231/2450] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:20:47.987000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.046010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1756c2ee-3710-4103-b344-3076189e9e57 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:48.051086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:48.051696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.051974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.052243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] PATH_INFO: `/v3/users/25ed02f0e7654f0493a77ecea1285555` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.053364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ed02f0e7654f0493a77ecea1285555)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:48.055309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:48.099909 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ed02f0e7654f0493a77ecea1285555'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.108375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ed02f0e7654f0493a77ecea1285555'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.115988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ed02f0e7654f0493a77ecea1285555'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.116699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ed02f0e7654f0493a77ecea1285555'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.117381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ed02f0e7654f0493a77ecea1285555'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.134721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1220/2451] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:47 2022] DELETE /identity/v3/users/25ed02f0e7654f0493a77ecea1285555 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:48.139509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.139795 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.140069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.158611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:48.159796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:48.217424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1232/2452] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:48.227142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.228539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:48.243320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:48.286455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec9667-6219-4967-9c3b-241034d914c9 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:48.290066 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:48.290690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.290968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.291236 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.300376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:48.301784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d06b-84cd-4e53-86c7-500061ac32c3 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:48.310857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1221/2453] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:20:48.577059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.578485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:48.592969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56627e-b088-456a-a950-b6bf2bef9b6f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:48.598227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:48.598899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.599177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.599466 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] PATH_INFO: `/v3/projects/5405ae53c5c34c0fbd7a91d8ffc0ef3d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.600576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5405ae53c5c34c0fbd7a91d8ffc0ef3d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:48.602473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:48.648385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5405ae53c5c34c0fbd7a91d8ffc0ef3d', 'request_id': 'req-81fb4971-c28b-4598-882b-6a53bf00df96'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.653703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5405ae53c5c34c0fbd7a91d8ffc0ef3d', 'request_id': 'req-81fb4971-c28b-4598-882b-6a53bf00df96'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.660839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5405ae53c5c34c0fbd7a91d8ffc0ef3d', 'request_id': 'req-81fb4971-c28b-4598-882b-6a53bf00df96'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.671470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] The token cache is being invalidate because project 5405ae53c5c34c0fbd7a91d8ffc0ef3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:48.671750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.672366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-81fb4971-c28b-4598-882b-6a53bf00df96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:48.673816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1233/2454] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:48 2022] DELETE /identity/v3/projects/5405ae53c5c34c0fbd7a91d8ffc0ef3d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:48.679454 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.679902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.679996 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.698946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:48.700118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:48.756686 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1222/2455] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:48.820123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.820407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.820684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.840494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:48.841691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:48.850454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1234/2456] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:48.854645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.864427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702435f-77ca-4c3e-9a59-4a607947c0b4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:48.869182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:48.869801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.870067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.870338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.871027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:48.871332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:48.872922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:48.884773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1223/2457] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:48.889266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.899301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901db72-95ab-44eb-9eba-ef943cf53c4b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:48.904175 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:48.904714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:48.904995 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:48.905265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:48.905889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:48.907821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:48.934720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1235/2458] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:48 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:48.938759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:48.948902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7299d0-d2a9-4cac-8800-ab55944c50ce admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.002415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.003064 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.003346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.003623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.004251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.006211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.025827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1224/2459] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:48 2022] POST /identity/v3/users => generated 281 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:49.029827 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.040592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68663260-ab58-46d2-b935-5e69df416931 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.045201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.045798 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.046080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.046356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.047041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.048631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.058228 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1236/2460] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:49.061941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.072085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e99b-72ab-4152-bc41-80012358e407 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.077956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.078561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.078875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.079152 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.079793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.081920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c50130-3106-4e7f-b12c-539c56606a4a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.092490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1225/2461] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:49.097091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.107126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb23f7-8d10-42f7-9cfa-112016f03104 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.111737 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.112398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.112672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.112942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] PATH_INFO: `/v3/projects/63991fb427d94000a59b9eae2c054f0f/users/ae3df3db839c42a5a1881f2ad80d1081/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.127183 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63991fb427d94000a59b9eae2c054f0f, user_id=ae3df3db839c42a5a1881f2ad80d1081, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.130281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.142663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1237/2462] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:49 2022] PUT /identity/v3/projects/63991fb427d94000a59b9eae2c054f0f/users/ae3df3db839c42a5a1881f2ad80d1081/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:49.153097 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.153383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.153681 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.180040 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] MFA Rules not processed for user `ae3df3db839c42a5a1881f2ad80d1081`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:49.181157 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.189845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:49.243662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1226/2463] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:49.276564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.278245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.288899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:49.335004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7a70c-0288-4b9a-8507-2c59eb1584c8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.338608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.339200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.339478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.339746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.347736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.349135 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d76bb-056a-441d-854f-f0b9a62be8e0 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.357562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1238/2464] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:49.372333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.437472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246c68f-9ecb-43a9-9d7d-602d45d86139 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.442825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.443447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.443725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.443997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] PATH_INFO: `/v3/users/ae3df3db839c42a5a1881f2ad80d1081` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.445151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3df3db839c42a5a1881f2ad80d1081)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.447107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.490914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3df3db839c42a5a1881f2ad80d1081'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:49.497503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3df3db839c42a5a1881f2ad80d1081'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:49.504591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3df3db839c42a5a1881f2ad80d1081'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:49.505224 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3df3db839c42a5a1881f2ad80d1081'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:49.505838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3df3db839c42a5a1881f2ad80d1081'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:49.523242 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1227/2465] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:20:49 2022] DELETE /identity/v3/users/ae3df3db839c42a5a1881f2ad80d1081 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:49.527650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.527970 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.528250 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.546608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:49.547800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.605287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1239/2466] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:49.615840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.617238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.651797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:49.677128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f92d3-cdc1-4304-a024-ba61762dff0b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.680783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.681397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.681647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.681938 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.691044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.692720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86734e73-3033-4e9a-872f-4f81a0965977 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.701869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1228/2467] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:49.969361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.970858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.982687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423d687-97be-4bfd-afde-2e8cb2b1fafe None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.987342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.987985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.988269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.988548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] PATH_INFO: `/v3/projects/63991fb427d94000a59b9eae2c054f0f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.989677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63991fb427d94000a59b9eae2c054f0f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.991767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.037579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63991fb427d94000a59b9eae2c054f0f', 'request_id': 'req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:50.043095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63991fb427d94000a59b9eae2c054f0f', 'request_id': 'req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:50.050760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63991fb427d94000a59b9eae2c054f0f', 'request_id': 'req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:50.061425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] The token cache is being invalidate because project 63991fb427d94000a59b9eae2c054f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:20:50.061710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:50.062333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1dd0e977-7f3e-44b4-979d-3e0e4739ddc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:20:50.063848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1240/2468] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:20:49 2022] DELETE /identity/v3/projects/63991fb427d94000a59b9eae2c054f0f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:50.071113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.071399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.071684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.090908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:50.092100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:50.149543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1229/2469] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:50.157285 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.157576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.157882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.176803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:50.177986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:50.185765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1241/2470] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:50.189630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.199874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a3166-706a-4838-8790-0ed73d996b1b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.204437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.205029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.205311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.205584 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.206882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:20:50.207204 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.208767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.219929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1230/2471] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:50.223896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.233024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bb8aa-234f-415f-9f0c-cf5578a239a2 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.237591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.238178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.238464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.238772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.239407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.241282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.269296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1242/2472] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:50.273409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.283221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425aacab-c11b-42ea-ad3c-2540fb250348 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.337349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.337940 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.338215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.338484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.339121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.341016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.360813 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1231/2473] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:50 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:20:50.365473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.375604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf3aa3-ddcc-4973-94b7-66659b3055f2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.380820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.381408 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.381690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.381962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.382629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.384217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.385422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:50.398065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1243/2474] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:50.399231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.408932 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920ead7-a7d7-4f83-8f17-791c0ad011d7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.413561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.414148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.414427 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.414735 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.415374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.416952 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71444c3-d077-4ea5-a9a5-174b7123d17d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.427804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1232/2475] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:20:50.431880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.442272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84dd444-5f36-492e-9a9c-b444df7de8f8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.447109 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.447761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.448039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.448311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] PATH_INFO: `/v3/projects/5fa6be82e0004c208037e16b0bfc23a2/users/1734d2f354a247d680f3c6fe7071632c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.462351 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa6be82e0004c208037e16b0bfc23a2, user_id=1734d2f354a247d680f3c6fe7071632c, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.465005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.477211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1244/2476] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:50 2022] PUT /identity/v3/projects/5fa6be82e0004c208037e16b0bfc23a2/users/1734d2f354a247d680f3c6fe7071632c/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:50.482028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55588b-2dbe-4332-bb65-f6883bdb5026 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.482334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55588b-2dbe-4332-bb65-f6883bdb5026 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.482638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55588b-2dbe-4332-bb65-f6883bdb5026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.501634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cc55588b-2dbe-4332-bb65-f6883bdb5026 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:50.503462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cc55588b-2dbe-4332-bb65-f6883bdb5026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:50.535287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:50.561017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1233/2477] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:50.570926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:50.572604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:50.629839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769feb15-3816-484d-8186-a97cb8a6b408 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:50.633502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:50.634092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:50.634369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:50.634674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:50.644075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:50.645479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:50.654731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1245/2478] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:54.415986 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.416495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.416678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.446359 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] MFA Rules not processed for user `1734d2f354a247d680f3c6fe7071632c`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:54.447714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:54.515729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1234/2479] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:54.520245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.529758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8caae7-5a7c-4636-9f23-00a23afb765f service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.535119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.535713 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.535999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.536283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.536906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.538813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.564832 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1246/2480] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:20:54 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:54.569619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.584573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb25f75-e84e-4256-a5a0-33f442bf3eb9 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.665694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.666312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.666616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.666895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.667775 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.669750 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.694893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1235/2481] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:20:54 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:20:54.696447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.706602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7050efbc-ec4a-47cf-8279-fdc2811fadb7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.711894 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.712588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.712965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.713332 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.714055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.715765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.731467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1247/2482] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:20:54.732807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.743752 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb894d6-dc7f-4487-98a5-ef6a41dcc27f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.752260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.752868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.753153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.753494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] PATH_INFO: `/v3/projects/82d74d9e4d884937828d00033bcf5f63/users/8f86f16d3c1f4c1486ff45763e951de3/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.776968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d74d9e4d884937828d00033bcf5f63, user_id=8f86f16d3c1f4c1486ff45763e951de3, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.780080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.802346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1236/2483] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:54 2022] PUT /identity/v3/projects/82d74d9e4d884937828d00033bcf5f63/users/8f86f16d3c1f4c1486ff45763e951de3/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:20:54.803295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.817379 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb638941-75e3-4b16-81b5-b96abcfae8a6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.880982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.881623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.881899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.882167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.882852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.884434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbafb55-fedc-49f3-9abe-9273db6e97a4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.894895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1248/2484] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:20:54.899028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.911218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74095913-e926-40f4-b871-b744d6a568af admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.917451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.918147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.918433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.918781 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] PATH_INFO: `/v3/projects/82d74d9e4d884937828d00033bcf5f63/users/8f86f16d3c1f4c1486ff45763e951de3/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.930324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d74d9e4d884937828d00033bcf5f63, user_id=8f86f16d3c1f4c1486ff45763e951de3, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.933809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.949043 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1237/2485] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:20:54 2022] PUT /identity/v3/projects/82d74d9e4d884937828d00033bcf5f63/users/8f86f16d3c1f4c1486ff45763e951de3/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:20:59.244706 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4b34-b977-44d1-b360-53d5a562cd38 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:59.245197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4b34-b977-44d1-b360-53d5a562cd38 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:59.245197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4b34-b977-44d1-b360-53d5a562cd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:59.289920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d67d4b34-b977-44d1-b360-53d5a562cd38 None None] MFA Rules not processed for user `8f86f16d3c1f4c1486ff45763e951de3`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:59.291510 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d67d4b34-b977-44d1-b360-53d5a562cd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:59.309480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:59.411024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1249/2486] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:20:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:59.426283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:59.428374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:59.478295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:59.636884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952a6d7-ff23-426e-b5c2-0640009b0f47 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:59.650684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:59.659355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:59.659355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:59.659355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:59.749314 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:59.751376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2f67a-061e-477c-ad9c-01f3f7529d53 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:59.766957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1238/2487] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:21:00.737849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:00.738317 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:00.738538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:00.767384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] MFA Rules not processed for user `1734d2f354a247d680f3c6fe7071632c`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:21:00.770246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:00.779910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1250/2488] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:21:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:21:09.767911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=76 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:10.594673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:10.594964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:10.595238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:10.614911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] MFA Rules not processed for user `0a30897a10d8467aa294f24954368a84`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:21:10.616911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:10.673806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1239/2489] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:21:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:21:10.758549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=35 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:20.666073 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=25 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:25.206673 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:21:25.208238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:25.305809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ac9435-7455-4340-b226-cdc7348d7b10 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:21:25.310987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:21:25.311828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:25.312103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:25.312375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:25.314101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:25.326702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:21:25.328287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:21:25.338413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1251/2490] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:21:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:21:35.333968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=41 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:40.081941 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:21:40.083371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:40.160452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a09cf0-5772-4e23-8a51-ca4d0d71c07a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:21:40.163980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:21:40.165087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:40.165366 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:40.165633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:40.167403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:40.226155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:21:40.227616 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f19834-7751-4b6f-a1f3-896c959c603a service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:21:40.236606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1240/2491] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:21:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:21:41.372249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:21:41.373765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:41.450033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5be7ca-79d9-4c11-ae48-5f8c6abbc499 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:21:41.453675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:21:41.454272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:41.454546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:41.454843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:41.464144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:21:41.465542 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:21:41.474649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1252/2492] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:21:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:21:50.232513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=57 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:51.470625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=35 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:52.306462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449682d2-88c2-41cf-869b-f861c3f96d1c None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:21:52.307077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449682d2-88c2-41cf-869b-f861c3f96d1c None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:21:52.307184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449682d2-88c2-41cf-869b-f861c3f96d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:21:52.328353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-449682d2-88c2-41cf-869b-f861c3f96d1c None None] MFA Rules not processed for user `7ebc052a3cb6400a961c1b9d9bec83d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:21:52.329593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-449682d2-88c2-41cf-869b-f861c3f96d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:21:52.336644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1241/2493] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:21:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:02.321984 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:09.905986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:09.914893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a514-bf53-4110-864d-bb015affe0d1 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:09.918523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:09.919151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:09.919432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:09.919705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:09.921436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:09.986868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:09.988311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:09.998594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1253/2494] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:19.994139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=30 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:39.652183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:39.652722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:39.652833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:39.694053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:39.695365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:39.762336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1242/2495] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:39.813677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:39.815118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:39.886528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160317c-8ab7-45d6-bf0f-0baa521f3098 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:39.891928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:39.892501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:39.892780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:39.893051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:39.902261 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:39.903965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:39.912663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1254/2496] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:22:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:22:49.754786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=29 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:49.909804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=32 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:51.346551 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:51.356098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4144973-837b-4d40-b3ee-d43825462b26 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:51.359690 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:51.360356 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.360624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.360850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.362647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.449302 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:51.451085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:51.471116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1243/2497] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:55.707180 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:55.708831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:55.727188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a5042-9ff4-4c43-a1b0-22ef2154989f service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:55.732661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:55.733565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.733949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.734329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] PATH_INFO: `/v3/users/951cabf887f14896b0c4402b686be697` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.756482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=951cabf887f14896b0c4402b686be697)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:55.759327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:55.811574 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '951cabf887f14896b0c4402b686be697'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:55.818994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '951cabf887f14896b0c4402b686be697'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:55.828521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '951cabf887f14896b0c4402b686be697'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:55.829166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '951cabf887f14896b0c4402b686be697'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:55.829766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '951cabf887f14896b0c4402b686be697'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:55.849820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1255/2498] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:55 2022] DELETE /identity/v3/users/951cabf887f14896b0c4402b686be697 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:56.039338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.062044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97ce06-85dc-4a17-99fb-8784461f8f1e service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.097465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.099226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.172003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd849909-c6ec-497d-9bfe-341eb6e09fc6 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.177505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.178534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.179398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.179398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.182216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.271908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.273324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.284836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1256/2499] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:22:56.285734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.285875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.286731 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.287329 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.287476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.287615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] PATH_INFO: `/v3/projects/c66f397e080f497da06b661ea1a87e03` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.295430 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:56.303533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66f397e080f497da06b661ea1a87e03)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.307773 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.360456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f61216f-5893-497b-8ca8-fd980ce38742 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.383010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66f397e080f497da06b661ea1a87e03', 'request_id': 'req-14d9619c-846a-47dc-b406-3234430880fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:56.391254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66f397e080f497da06b661ea1a87e03', 'request_id': 'req-14d9619c-846a-47dc-b406-3234430880fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:56.393212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:56.401168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66f397e080f497da06b661ea1a87e03', 'request_id': 'req-14d9619c-846a-47dc-b406-3234430880fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:56.418838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] The token cache is being invalidate because project c66f397e080f497da06b661ea1a87e03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:56.419132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:56.419793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-14d9619c-846a-47dc-b406-3234430880fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:56.421518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1244/2500] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:56 2022] DELETE /identity/v3/projects/c66f397e080f497da06b661ea1a87e03 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:56.425252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.425927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.426394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.427253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.427672 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.428160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.428631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.429844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.508267 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:56.510196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.517545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.519263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73b0752-fa08-4a58-b88d-c874e1e943e4 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.533539 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1257/2501] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:56.589595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1245/2502] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:56.597511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.597994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.598357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.623862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:56.625304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.635565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1258/2503] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:56.640715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.653511 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0a94-ff68-44da-98a5-26c036034f5d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.660759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.661492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.661835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.662201 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.662990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:56.663388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.673289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.690203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1246/2504] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:56.694150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.705742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc7bb6-6db5-48f4-9bb0-eee9010d88f7 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.711549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.712775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.713018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.713256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.713899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.715856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.782629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1259/2505] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:56.785721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.802778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748fd4b-1386-406a-a82d-e1fd727d5e98 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.895580 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.896364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.896654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.896928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.897576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.900373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.960485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1247/2506] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:22:56.964735 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.981678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c94291-2e60-4c12-acb0-4e92ac759d17 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.989563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.990284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.990734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.991600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.992862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.994762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.008076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1260/2507] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:57.013301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:57.028250 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbd215-362c-4578-867c-98cc3ec05f98 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:57.033182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:57.033789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.034149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.034455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.035128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:57.036721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8b69e-65e6-49da-be71-59484a9899b7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.051070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1248/2508] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:57.051821 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:57.062554 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab44fcf-7651-4548-a927-91e575656e93 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:57.068409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:57.069083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.069352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.069627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] PATH_INFO: `/v3/projects/92fa9dd2388b485195253752f4afa47f/users/382570a5f51d4f29ae4faff617a144d3/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.079826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:57.089319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92fa9dd2388b485195253752f4afa47f, user_id=382570a5f51d4f29ae4faff617a144d3, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:57.092440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.112456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1261/2509] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:57 2022] PUT /identity/v3/projects/92fa9dd2388b485195253752f4afa47f/users/382570a5f51d4f29ae4faff617a144d3/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:57.132540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c4b3-7879-459d-89be-6ed2742b58b0 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.132900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c4b3-7879-459d-89be-6ed2742b58b0 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.133245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c4b3-7879-459d-89be-6ed2742b58b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.167555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0ca2c4b3-7879-459d-89be-6ed2742b58b0 None None] MFA Rules not processed for user `382570a5f51d4f29ae4faff617a144d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:57.169085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0ca2c4b3-7879-459d-89be-6ed2742b58b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:57.179364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:57.241448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1249/2510] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:57.255008 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:57.256911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:57.340431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad6b78-25a4-4d36-9389-552247e46358 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:57.346809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:57.347479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.347770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.348104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.358623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:57.360481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.372156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1262/2511] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:04.263473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:04.263967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:04.264063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:04.317339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:04.318567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:04.406877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1250/2512] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:23:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:04.417656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:04.419155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:04.521622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fafaa-108c-46ca-b514-d1058f2082f4 service nova] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:04.528045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:04.528659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:04.528945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:04.529216 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:04.539796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:04.541413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747aa16-c5ba-475d-a835-9917b7c3d44c service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:04.550385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1263/2513] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:23:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:23:07.032176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.106238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d092824-b2b1-4b6a-be2b-9cd9877c0e20 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.115903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.117471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.118208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.119380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] PATH_INFO: `/v3/users/382570a5f51d4f29ae4faff617a144d3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.121521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382570a5f51d4f29ae4faff617a144d3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.124115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.153933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:07.186824 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382570a5f51d4f29ae4faff617a144d3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.196074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382570a5f51d4f29ae4faff617a144d3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.207104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382570a5f51d4f29ae4faff617a144d3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.207815 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382570a5f51d4f29ae4faff617a144d3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.208448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382570a5f51d4f29ae4faff617a144d3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.233707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1251/2514] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:07 2022] DELETE /identity/v3/users/382570a5f51d4f29ae4faff617a144d3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:07.239229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.239823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.239823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.264836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:07.264836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.311310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:07.340696 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1264/2515] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:07.355930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.357725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.432856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d124357-9a46-4adb-ab14-fe7015d87bf0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.436501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.437108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.437385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.437653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.448420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.449844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62519849-ec1c-47d7-bbac-fa3e234cfee6 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.459312 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1252/2516] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:07.826016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.826449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.842139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9fb451-e52f-47aa-a23b-d5db8694d8f3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.846919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.847575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.847855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.848144 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] PATH_INFO: `/v3/projects/92fa9dd2388b485195253752f4afa47f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.849297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92fa9dd2388b485195253752f4afa47f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.854738 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.926469 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92fa9dd2388b485195253752f4afa47f', 'request_id': 'req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.936772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92fa9dd2388b485195253752f4afa47f', 'request_id': 'req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.948084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92fa9dd2388b485195253752f4afa47f', 'request_id': 'req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.959871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] The token cache is being invalidate because project 92fa9dd2388b485195253752f4afa47f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:07.960159 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.960814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-1a34a1a7-4f8f-4702-8d62-8dc36f41c154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.962364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1265/2517] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:07 2022] DELETE /identity/v3/projects/92fa9dd2388b485195253752f4afa47f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:07.970205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.970493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.970807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.990372 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:07.991614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.073518 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1253/2518] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:08.083768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.084051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.084316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.106483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:08.107747 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.116118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1266/2519] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:08.121369 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.132092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ec24d5-9a7f-4d9c-bf71-41af85528e47 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.137403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.138125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.138422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.138806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.139476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:08.139774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.141642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.155153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1254/2520] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:08.159582 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.172256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e054-099c-42d2-afbd-40398e275b5c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.178642 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.179338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.179600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.179867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.180524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.184528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.234295 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1267/2521] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:08.239754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.252823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cf944-ba5d-428a-8a81-53a96d80d964 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.297131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:08.331321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.331934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.332222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.332532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.333206 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.335169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.391041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1255/2522] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:08.396456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.412137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a452-1c35-4b4f-8915-c9d45fb876a2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.417740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.418470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.418843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.419186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.420074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.422088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.434903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1268/2523] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:08.440887 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.457189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000f0070-9f21-41d3-96b5-614e7698fdca admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.463840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.464845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.465398 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.466074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.467455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.469541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3ef73-2b8c-46fe-ac7b-675b625968c1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.483702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1256/2524] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:08.487886 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.501269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4998ee60-9343-4e37-b1b4-21b4fc0280b4 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.506046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.506666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.506947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.507220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] PATH_INFO: `/v3/projects/a9e5c63ce58a48ae8db07510dca36726/users/500d05a4db3a44aaa978cd81af6a2446/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.525172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e5c63ce58a48ae8db07510dca36726, user_id=500d05a4db3a44aaa978cd81af6a2446, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.527998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.541890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1269/2525] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:08 2022] PUT /identity/v3/projects/a9e5c63ce58a48ae8db07510dca36726/users/500d05a4db3a44aaa978cd81af6a2446/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:08.552426 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.552707 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.552985 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.582183 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] MFA Rules not processed for user `500d05a4db3a44aaa978cd81af6a2446`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:08.583513 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.685146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1257/2526] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:08.689982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.756987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef5b569-b95e-4060-aa3c-86c07e9c2be2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.761683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.762289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.762716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.762931 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.763570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.765529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.771756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:08.802136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1270/2527] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:08.806923 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.820107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19aa9bc-944c-49ca-90e4-f0f7dba8a6ca None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.915243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.915963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.916315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.917096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.917420 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.919738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.946685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1258/2528] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:08 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:08.951325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.962019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0af09-37d3-40f2-b1e5-94d4d453df41 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.968114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.968787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.969131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.969516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.970259 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.972139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.984053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1271/2529] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:08.988539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.002918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d6334-d59c-40c9-9025-cc7ef95a84f1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.010353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.011431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.011853 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.012316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] PATH_INFO: `/v3/projects/1d84d708048a43b6ba3ac01d04416158/users/e999908534b54f898dfdd46f19e30646/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.034603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d84d708048a43b6ba3ac01d04416158, user_id=e999908534b54f898dfdd46f19e30646, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.037524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.058158 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1259/2530] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:08 2022] PUT /identity/v3/projects/1d84d708048a43b6ba3ac01d04416158/users/e999908534b54f898dfdd46f19e30646/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:09.062449 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.076263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f267694e-09b1-450f-b159-32e80e2e7f71 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.137523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.138512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.139007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.139307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.140031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.141586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b88c17-e14b-47b8-99e4-950c1a8291de admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.154841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1272/2531] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:09.158963 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.170149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c781c9e-6faa-43a9-ace2-1e560f21906d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.175981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.176721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.177176 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.177377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] PATH_INFO: `/v3/projects/1d84d708048a43b6ba3ac01d04416158/users/e999908534b54f898dfdd46f19e30646/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.180296 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d84d708048a43b6ba3ac01d04416158, user_id=e999908534b54f898dfdd46f19e30646, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.185266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.189375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:09.206404 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1260/2532] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:09 2022] PUT /identity/v3/projects/1d84d708048a43b6ba3ac01d04416158/users/e999908534b54f898dfdd46f19e30646/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:09.226858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.227322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.228382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.264043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] MFA Rules not processed for user `e999908534b54f898dfdd46f19e30646`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:09.265266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.338831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1273/2533] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:09.350368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.352140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.418649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ca82d-f17d-4309-91de-3b8e6c1c974f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.423825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.424639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.424913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.425185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.435946 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.437701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.448759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1261/2534] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:09.526313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.537323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9e63b-44b4-403c-addf-39417b23f84a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.541618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.542215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.542492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.542800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.611074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:09.615457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.619121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50469e10-86a1-442e-bb02-cf1c31311ab5 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.646646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1274/2535] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:19.443461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:19.631299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:24.188663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:24.190185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.265886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d516fb-e631-441f-a005-ff11f2fb73aa service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:24.269812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:24.270929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:24.271379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:24.271821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:24.273845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.287968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:24.289467 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:24.297684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1262/2536] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:31.231760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.232677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.233560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.257355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:31.258767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:31.321387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1275/2537] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:31.336148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.338440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:31.431726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05e91-a1f2-412a-9346-cc7481117c79 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.437662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.438821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.439310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.439840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.451590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.453507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.464414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1263/2538] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:23:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:23:36.970440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.971835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:36.983817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2090508-379e-4c20-a6ba-8c734d4d5b25 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.987352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.987951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.988235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.988507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.990171 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:37.051422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:37.053139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.062927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1276/2539] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:40.068533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.070244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:40.087386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3303a7-02fa-43c5-adfa-bc0a4daeace7 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.092526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.093174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.093453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.093727 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] PATH_INFO: `/v3/users/0a30897a10d8467aa294f24954368a84` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.109386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a30897a10d8467aa294f24954368a84)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.111385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.158895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a30897a10d8467aa294f24954368a84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.165880 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a30897a10d8467aa294f24954368a84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.174470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a30897a10d8467aa294f24954368a84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.175167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a30897a10d8467aa294f24954368a84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.175765 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-91176a27-1473-425e-a285-188ac37d432d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a30897a10d8467aa294f24954368a84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.188642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:40.197284 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1264/2540] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:40 2022] DELETE /identity/v3/users/0a30897a10d8467aa294f24954368a84 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:40.372015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.391828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eea87-0b04-4212-a1e1-eca65184200d service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.453357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.453984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.454264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.454537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] PATH_INFO: `/v3/projects/47d9cb4aa3484f95bad54f2233a08439` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.464611 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47d9cb4aa3484f95bad54f2233a08439)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.466524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.499933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:40.513488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47d9cb4aa3484f95bad54f2233a08439', 'request_id': 'req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.518865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47d9cb4aa3484f95bad54f2233a08439', 'request_id': 'req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.526902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47d9cb4aa3484f95bad54f2233a08439', 'request_id': 'req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.538294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] The token cache is being invalidate because project 47d9cb4aa3484f95bad54f2233a08439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:40.538597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.539237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cc3a5a4e-5fb2-4c6e-a743-2c4ae9321548 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:40.541301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1277/2541] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:40 2022] DELETE /identity/v3/projects/47d9cb4aa3484f95bad54f2233a08439 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:40.546187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.546475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.546780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.565395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:40.566595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:40.623826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1265/2542] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:40.632188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.632472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.632742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.653451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:40.654717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:40.662767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1278/2543] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:40.668016 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.678301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6c0ad-aabe-443a-b4ea-8d6945d132dd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.683756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.684529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.684947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.685235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.686030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:40.686613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.688338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.702944 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1266/2544] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:40.707394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.717099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c579e-e3f0-447a-80aa-f219aa14b9f1 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.721677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.722263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.722538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.722838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.723467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.725327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.752831 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1279/2545] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:40 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:40.756976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.767462 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6ed8-57ec-443c-903b-0f2ffb511c69 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.821788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.822562 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.822886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.823313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.823947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.826149 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.852480 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1267/2546] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:40 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:40.858275 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.871483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ef0dd-ec7a-40b0-8424-c24082a51a4e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.876623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.877239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.877895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.877895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.878422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.880069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.891809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1280/2547] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:40.895800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.905919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce5f68-e047-4783-863b-a72e7833a807 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.910720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.911326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.911609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.911883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.912528 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.914137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cc91a-5c0b-494e-971b-a0922838fe84 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.925678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1268/2548] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:40.929626 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:40.940084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519efd96-52d2-4b9e-b87c-f684dc518d13 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:40.944768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:40.945521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.945813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.946091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] PATH_INFO: `/v3/projects/467f8310c7784773a07b2ac810316ef0/users/f82260a02326446db102c85ec3de0640/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:40.962075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=467f8310c7784773a07b2ac810316ef0, user_id=f82260a02326446db102c85ec3de0640, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:40.965487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:40.979815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1281/2549] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:40 2022] PUT /identity/v3/projects/467f8310c7784773a07b2ac810316ef0/users/f82260a02326446db102c85ec3de0640/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:40.989825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:40.990110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:40.990383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:41.017560 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] MFA Rules not processed for user `f82260a02326446db102c85ec3de0640`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:41.018810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:41.080483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:41.100774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1269/2550] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:41.114252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:41.115980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:41.188303 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512df243-5a18-49bf-8f07-dbee0b2509b2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:41.194071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:41.194669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:41.194946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:41.195211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:41.203648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:41.205201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ce78d-0588-434c-8a8f-224f66bb2459 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:41.214544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1282/2551] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:48.237970 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:48.300896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f086ef-e086-4f8b-ac35-f24df9cd4279 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:48.305610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:48.306937 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.307221 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.307495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] PATH_INFO: `/v3/users/f82260a02326446db102c85ec3de0640` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.308629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82260a02326446db102c85ec3de0640)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:48.310630 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:48.357956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82260a02326446db102c85ec3de0640'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.364576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82260a02326446db102c85ec3de0640'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.373365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82260a02326446db102c85ec3de0640'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.373976 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82260a02326446db102c85ec3de0640'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.374645 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82260a02326446db102c85ec3de0640'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.392444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1270/2552] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:48 2022] DELETE /identity/v3/users/f82260a02326446db102c85ec3de0640 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:48.396936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.397418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.397526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.416399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:48.417603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.464973 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:48.488712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1283/2553] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:48.498702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:48.500125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.557524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad1e0e-d730-4ae2-bc42-af8ce8d311ed admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:48.561140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:48.561683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.561968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.562225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.572653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:48.574788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ab18a-2eb0-444f-bb44-5fbf698677f0 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:48.586022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1271/2554] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:48.881926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:48.883396 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.895292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f54f9a-ef29-4254-b273-ce851c83a162 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:48.900463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:48.901103 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.901383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.901686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] PATH_INFO: `/v3/projects/467f8310c7784773a07b2ac810316ef0` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.902913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=467f8310c7784773a07b2ac810316ef0)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:48.905112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:48.954502 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '467f8310c7784773a07b2ac810316ef0', 'request_id': 'req-106fe284-56f8-4da4-ab89-4016bebe3c8b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.960044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '467f8310c7784773a07b2ac810316ef0', 'request_id': 'req-106fe284-56f8-4da4-ab89-4016bebe3c8b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.967905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '467f8310c7784773a07b2ac810316ef0', 'request_id': 'req-106fe284-56f8-4da4-ab89-4016bebe3c8b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.980149 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] The token cache is being invalidate because project 467f8310c7784773a07b2ac810316ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:48.980440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.981166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-106fe284-56f8-4da4-ab89-4016bebe3c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.982689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1284/2555] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:48 2022] DELETE /identity/v3/projects/467f8310c7784773a07b2ac810316ef0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:48.988631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.988949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.989222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.010177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:49.011383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:49.016523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:49.083300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1272/2556] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:49.092270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.092560 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.092840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.114329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:49.115572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:49.124579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1285/2557] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:49.128895 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.138724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0453e63-5b4e-4fbe-bfb7-794e2aec70be None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.143380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.144041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.144245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.144519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.145181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:49.145485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.147113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.159309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1273/2558] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:49.163442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.175595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1ab83-2da8-438f-8726-f29396e3a28e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.182338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.182813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.183094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.183366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.184007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.186088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.221530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1286/2559] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:49.223029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.234990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7874eb3-2362-4c0c-a1e4-45cfebc543a7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.305568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.306939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.307181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.307517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.308233 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.310862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.334939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1274/2560] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:49.339196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.355708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff3090-c631-4dec-baaf-b30790d9f97c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.360682 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.361298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.361576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.361851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.362505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.364188 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.382421 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1287/2561] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:49.383945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.396306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee3114-6a05-41d8-8c81-ce8896b7b1f9 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.402536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.403175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.403458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.403914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.404563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.406192 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ec6f3-be64-495d-a667-52d5020a7915 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.423144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1275/2562] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:49.424304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.438816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e4a66-4bf0-4264-89c7-b2956fa33923 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.444084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.444752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.445043 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.445439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] PATH_INFO: `/v3/projects/2b75b363fab94b4fa3b11601114202c4/users/4be02f59083e4c50aa5f54d386c198de/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.464479 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b75b363fab94b4fa3b11601114202c4, user_id=4be02f59083e4c50aa5f54d386c198de, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.468184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.484813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1288/2563] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:49 2022] PUT /identity/v3/projects/2b75b363fab94b4fa3b11601114202c4/users/4be02f59083e4c50aa5f54d386c198de/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:49.502628 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.502914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.503244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.534273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] MFA Rules not processed for user `4be02f59083e4c50aa5f54d386c198de`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:49.535644 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:49.606600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1276/2564] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:49.612465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.681089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe24d0f-405e-4f66-8fa1-cd8820fd9cf3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.687069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.687680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.687962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.688237 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.688871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.690800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.696444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:49.721983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1289/2565] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:49.726322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.738039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c14760-8dca-4f96-adb7-0952b7b76341 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.742407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:49.810231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.811013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.811374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.811689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.812422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.814414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.838683 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1277/2566] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:49 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:49.844256 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.857640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebdc01-8f04-4024-8b58-2a328d17dc08 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.862544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.863193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.863478 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.863807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.864471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.866100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.876909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1290/2567] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:49.881215 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.892006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec099-fc92-42c0-9be3-97247998974b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.896795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.897451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.897730 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.898002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] PATH_INFO: `/v3/projects/e2e2511a039d4aada94bc64662df2af3/users/1f868fabedfb42cc94e0ba0876f82f99/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.913403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e2511a039d4aada94bc64662df2af3, user_id=1f868fabedfb42cc94e0ba0876f82f99, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.916579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.929309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1278/2568] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:49 2022] PUT /identity/v3/projects/e2e2511a039d4aada94bc64662df2af3/users/1f868fabedfb42cc94e0ba0876f82f99/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:49.934019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.944111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10de2d-4fb7-4e7c-a242-e8c95ca97251 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.001730 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.002336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.002702 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.002977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.003623 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.005198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ce738-1ca8-4fd5-bddd-75602d1d2853 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.015610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1291/2569] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:50.020363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.030926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64354949-d004-4a9e-8777-c9a23589c3b0 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.035631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.036338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.036564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.036837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] PATH_INFO: `/v3/projects/e2e2511a039d4aada94bc64662df2af3/users/1f868fabedfb42cc94e0ba0876f82f99/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.038878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e2511a039d4aada94bc64662df2af3, user_id=1f868fabedfb42cc94e0ba0876f82f99, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.042044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.053395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1279/2570] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:50 2022] PUT /identity/v3/projects/e2e2511a039d4aada94bc64662df2af3/users/1f868fabedfb42cc94e0ba0876f82f99/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:50.069876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74104840-a759-48bb-b07b-0a502a838719 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.070162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74104840-a759-48bb-b07b-0a502a838719 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.070523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74104840-a759-48bb-b07b-0a502a838719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.099778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-74104840-a759-48bb-b07b-0a502a838719 None None] MFA Rules not processed for user `1f868fabedfb42cc94e0ba0876f82f99`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:50.101001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-74104840-a759-48bb-b07b-0a502a838719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.171294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1292/2571] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:50.182618 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.184391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.242324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d51d4-b910-442e-a2be-0ffedf9ebe6e admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.246431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.247057 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.247336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.247605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.256607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.258164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f46c2e-7e5f-48d0-88ca-f8352f3e2aab service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.268720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1280/2572] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:50 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:50.667910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74104840-a759-48bb-b07b-0a502a838719 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.737107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:50.743483 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74104840-a759-48bb-b07b-0a502a838719 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.750643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.751477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.751824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.752172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] PATH_INFO: `/v3/users/4be02f59083e4c50aa5f54d386c198de` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.753442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be02f59083e4c50aa5f54d386c198de)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.755457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.817168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be02f59083e4c50aa5f54d386c198de'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.827776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be02f59083e4c50aa5f54d386c198de'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.840022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be02f59083e4c50aa5f54d386c198de'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.840942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be02f59083e4c50aa5f54d386c198de'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.841659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be02f59083e4c50aa5f54d386c198de'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.874077 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1293/2573] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:50 2022] DELETE /identity/v3/users/4be02f59083e4c50aa5f54d386c198de => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:50.879982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.880488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.880649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.902811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:50.902811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.967205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1281/2574] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:50.976974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.979053 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:51.036233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d40e5-4b35-4d1f-98fb-3ae3db1fc44c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.039777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.040375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.041260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.041534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.050647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.052058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.061398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1294/2575] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:51.333014 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.334780 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:51.342165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:51.351402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93d08f-619b-46bf-8970-0fb8f00011e5 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.356569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.357205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.357488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.357762 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] PATH_INFO: `/v3/projects/2b75b363fab94b4fa3b11601114202c4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.358928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b75b363fab94b4fa3b11601114202c4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.361003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.407832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b75b363fab94b4fa3b11601114202c4', 'request_id': 'req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.413235 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b75b363fab94b4fa3b11601114202c4', 'request_id': 'req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.424450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b75b363fab94b4fa3b11601114202c4', 'request_id': 'req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.436920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] The token cache is being invalidate because project 2b75b363fab94b4fa3b11601114202c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:51.437216 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.437864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.439382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1282/2576] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:51 2022] DELETE /identity/v3/projects/2b75b363fab94b4fa3b11601114202c4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:51.443368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.453692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033460f7-57a0-4742-8540-dbd2cf38dee5 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.510230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.510839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.511124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.511399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] PATH_INFO: `/v3/users/1f868fabedfb42cc94e0ba0876f82f99` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.512900 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f868fabedfb42cc94e0ba0876f82f99)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.515488 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.539354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:51.565446 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f868fabedfb42cc94e0ba0876f82f99'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.573012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f868fabedfb42cc94e0ba0876f82f99'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.580633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f868fabedfb42cc94e0ba0876f82f99'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.581244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f868fabedfb42cc94e0ba0876f82f99'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.581840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-457d2218-7322-49d2-9618-77a5f7ca2fe7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f868fabedfb42cc94e0ba0876f82f99'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:51.599057 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1295/2577] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:51 2022] DELETE /identity/v3/users/1f868fabedfb42cc94e0ba0876f82f99 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:51.877432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.879091 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:51.897476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c9b0-6e05-4f08-9240-9bf08c6f8eb1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.964110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.964747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.965030 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.965332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] PATH_INFO: `/v3/projects/e2e2511a039d4aada94bc64662df2af3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.966450 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e2511a039d4aada94bc64662df2af3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.968508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.024561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e2511a039d4aada94bc64662df2af3', 'request_id': 'req-829f6395-3c71-47bd-94bc-9db89d7457a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:52.031474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e2511a039d4aada94bc64662df2af3', 'request_id': 'req-829f6395-3c71-47bd-94bc-9db89d7457a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:52.038920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e2511a039d4aada94bc64662df2af3', 'request_id': 'req-829f6395-3c71-47bd-94bc-9db89d7457a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:52.051579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] The token cache is being invalidate because project e2e2511a039d4aada94bc64662df2af3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:52.051800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:52.052434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-829f6395-3c71-47bd-94bc-9db89d7457a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:52.053902 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1283/2578] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:51 2022] DELETE /identity/v3/projects/e2e2511a039d4aada94bc64662df2af3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:52.059335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.059805 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.060240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.080800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:52.081998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:52.162023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1296/2579] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:52.164268 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.166229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.166586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.186527 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:52.187757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:52.195793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1284/2580] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:52.201048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.210762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe92b29-5c34-4d68-9b68-c9b7f7d1f4ab None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.216051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.216647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.216927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.217202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.217848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:52.218152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.219771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.231427 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1297/2581] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:52.235913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.245239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee448f-4381-4965-af86-4019a7990fcc None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.249793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.250373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.250680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.250949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.251588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.253438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.258563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:52.282340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1285/2582] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:52.286531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.297042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580a7f9-5dc7-41b4-b223-3678e484c954 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.352902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.353938 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.354244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.354513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.355209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.357112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.378112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1298/2583] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:52.382154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.392489 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46958cb6-263a-47cb-9d3f-00858fad0ab1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.397173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968208-5613-4274-8b29-69898869b469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.397771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968208-5613-4274-8b29-69898869b469 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.398052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968208-5613-4274-8b29-69898869b469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.398322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968208-5613-4274-8b29-69898869b469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.399001 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968208-5613-4274-8b29-69898869b469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.401088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968208-5613-4274-8b29-69898869b469 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.411063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1286/2584] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:52.415132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.425442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61184f1-f94d-4e07-85a7-19df5f724322 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.429997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.430602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.430893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.431166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.432310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.433877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3156802-b1b1-49bd-b724-563a23d60ac3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.444965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1299/2585] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:52.450395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968208-5613-4274-8b29-69898869b469 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.460603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968208-5613-4274-8b29-69898869b469 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.465187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.465830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.466110 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.466376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] PATH_INFO: `/v3/projects/0b6a13747bea4ea9aba61b67c2715b8c/users/458b41cfb34548a082c1ab01415e8502/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.483162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6a13747bea4ea9aba61b67c2715b8c, user_id=458b41cfb34548a082c1ab01415e8502, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.485826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.500227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1287/2586] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:52 2022] PUT /identity/v3/projects/0b6a13747bea4ea9aba61b67c2715b8c/users/458b41cfb34548a082c1ab01415e8502/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:52.511205 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.511487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.511758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.519660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:52.539751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] MFA Rules not processed for user `458b41cfb34548a082c1ab01415e8502`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:52.540961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:52.606197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1300/2587] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:52.610539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.672530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ab19d-fdbc-42ea-870e-10ca12456e70 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.677342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.677945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.678228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.678512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.679180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.681105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.709361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1288/2588] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:52.715849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.728453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bf19a-26a0-46d6-8487-38a5e3cff24c None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.784839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.785448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.785727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.785999 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.786669 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.788551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.811083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1301/2589] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:52.815129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.825680 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f6969-05ac-470d-baa5-eef06807a004 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.830674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.831277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.831578 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.831837 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.832478 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.834074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.844782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1289/2590] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:52.848890 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.859612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c7bf5-5b91-40f7-8740-62402318bc4b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.864212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.864817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.865096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.865366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.866007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.867610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316f6923-76ff-4531-b021-dcf374d5f65d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.878357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1302/2591] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:52.882537 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:52.892661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a768671-bbd9-4a24-9a5b-5da6135dbfa6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:52.897237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:52.897876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.898409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.898409 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] PATH_INFO: `/v3/projects/524028509158417e96004b8de2da95cf/users/32e8775b67484e0096f01a7fbfc9c21f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.913539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=524028509158417e96004b8de2da95cf, user_id=32e8775b67484e0096f01a7fbfc9c21f, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:52.916724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:52.929823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1290/2592] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:52 2022] PUT /identity/v3/projects/524028509158417e96004b8de2da95cf/users/32e8775b67484e0096f01a7fbfc9c21f/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:52.941758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:52.942047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:52.942324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:52.970851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] MFA Rules not processed for user `32e8775b67484e0096f01a7fbfc9c21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:52.972061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:53.035177 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1303/2593] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:53.039619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.101573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872d86f-3f07-4b0f-a075-066ef687a00b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.106676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.107273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.107548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.107818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.108471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.110428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.115875 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:53.137807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1291/2594] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:53 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:53.141869 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.147365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:53.154480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ca06e2-855e-416f-924c-f89269845d50 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.211229 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.211844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.212126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.212399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.213565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.215477 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.237751 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1304/2595] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:53 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:53.241728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.251823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316fca8-b8ab-4967-887e-c402043885fd admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.256387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.256981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.257257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.257524 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.258155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.259896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.270034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1292/2596] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:53.274349 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.285147 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5ca4c-e99f-4d05-98a2-f522fea89cea admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.289851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.290501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.291473 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.291755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] PATH_INFO: `/v3/projects/c6f8f0b48e72462fa61a97772669b7c8/users/547210fc602945cebe18f8b1d31d74a5/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.307908 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f8f0b48e72462fa61a97772669b7c8, user_id=547210fc602945cebe18f8b1d31d74a5, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.310549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.323113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1305/2597] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:53 2022] PUT /identity/v3/projects/c6f8f0b48e72462fa61a97772669b7c8/users/547210fc602945cebe18f8b1d31d74a5/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:53.327893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.338049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5854d1-9e44-4c14-8585-e6ff7c68ecee admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.393654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.394182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.394463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.394778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.395422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.396997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99861ac5-e3ff-4634-a2e1-a72adb15b977 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.408319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1293/2598] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:53.412472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.424191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb97a7-97ad-42b2-a77a-bcb11af27fc3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.428919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.429576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.429849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.430127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] PATH_INFO: `/v3/projects/c6f8f0b48e72462fa61a97772669b7c8/users/547210fc602945cebe18f8b1d31d74a5/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.432114 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f8f0b48e72462fa61a97772669b7c8, user_id=547210fc602945cebe18f8b1d31d74a5, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.434928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.445583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1306/2599] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:53 2022] PUT /identity/v3/projects/c6f8f0b48e72462fa61a97772669b7c8/users/547210fc602945cebe18f8b1d31d74a5/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:53.460417 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.460751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.461022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.492303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] MFA Rules not processed for user `547210fc602945cebe18f8b1d31d74a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:53.493495 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:53.563759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1294/2600] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:23:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:53.575754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.577692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:53.637923 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3beb3-3cf4-4d02-8810-bf3527e97cd8 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.641884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.642491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.642791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.643062 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.699522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.701189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.709836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1307/2601] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:58.378980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.390360 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424631e-61ff-4a5e-af6d-638cdfb5756b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.393967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.394568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.394883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.395153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.449695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:58.452515 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.454746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.464506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1295/2602] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:59.054451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.063278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0896005-51aa-491a-9372-372d7027b3bb service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.066941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.067537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.067820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.068095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.075787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:59.080695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.082504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59125a20-9295-4d10-aa66-fff5b51f62a5 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.093391 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1308/2603] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:00.408005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:00.409846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:00.486237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f64e5-4602-459f-a957-3d49efbe7d2b service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:00.489625 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:00.490270 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:00.490555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:00.490863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] PATH_INFO: `/v3/users/500d05a4db3a44aaa978cd81af6a2446` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:00.492015 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=500d05a4db3a44aaa978cd81af6a2446)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:00.494458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:00.547059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500d05a4db3a44aaa978cd81af6a2446'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.555833 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500d05a4db3a44aaa978cd81af6a2446'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.564696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500d05a4db3a44aaa978cd81af6a2446'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.565309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500d05a4db3a44aaa978cd81af6a2446'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.565916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500d05a4db3a44aaa978cd81af6a2446'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.583218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1296/2604] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:00 2022] DELETE /identity/v3/users/500d05a4db3a44aaa978cd81af6a2446 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:00.587705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:00.588201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:00.588300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:00.607968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:00.609172 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:00.680778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1309/2605] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:00.692170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:00.693676 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:00.749857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef7bca-2303-48af-a855-d488965b48b3 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:00.753459 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:00.754058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:00.754338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:00.754639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:00.764397 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:00.765874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:00.776037 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1297/2606] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:01.054170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.055871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:01.068001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946ac8-6563-44ee-bf11-f8b2c64fae40 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.073352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.074010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.074314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.074609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] PATH_INFO: `/v3/projects/a9e5c63ce58a48ae8db07510dca36726` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.076025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e5c63ce58a48ae8db07510dca36726)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.077950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.125398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e5c63ce58a48ae8db07510dca36726', 'request_id': 'req-2f055410-3fa0-49a6-a76e-741a6421d1d0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.130971 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e5c63ce58a48ae8db07510dca36726', 'request_id': 'req-2f055410-3fa0-49a6-a76e-741a6421d1d0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.141495 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e5c63ce58a48ae8db07510dca36726', 'request_id': 'req-2f055410-3fa0-49a6-a76e-741a6421d1d0'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.154087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] The token cache is being invalidate because project a9e5c63ce58a48ae8db07510dca36726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:01.154376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.155929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.157442 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1310/2607] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:01 2022] DELETE /identity/v3/projects/a9e5c63ce58a48ae8db07510dca36726 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:01.161490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.171357 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c62e46-31ce-423d-83aa-597c97088c24 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.183418 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:01.228960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.229585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.229863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.230132 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] PATH_INFO: `/v3/users/e999908534b54f898dfdd46f19e30646` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.231254 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e999908534b54f898dfdd46f19e30646)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.233154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.281490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e999908534b54f898dfdd46f19e30646'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.290077 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e999908534b54f898dfdd46f19e30646'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.298921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e999908534b54f898dfdd46f19e30646'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.299828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e999908534b54f898dfdd46f19e30646'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.300523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a1b10aeb-7a1b-457d-a454-2b3b1b6954b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e999908534b54f898dfdd46f19e30646'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.319117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1298/2608] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:01 2022] DELETE /identity/v3/users/e999908534b54f898dfdd46f19e30646 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:01.613584 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.620010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:01.642759 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f055410-3fa0-49a6-a76e-741a6421d1d0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.707130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.707776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.708055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.708326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] PATH_INFO: `/v3/projects/1d84d708048a43b6ba3ac01d04416158` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.709461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d84d708048a43b6ba3ac01d04416158)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.711405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.735765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:01.765209 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d84d708048a43b6ba3ac01d04416158', 'request_id': 'req-e30c0d18-f067-4fb0-966e-c042151adbb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.772272 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d84d708048a43b6ba3ac01d04416158', 'request_id': 'req-e30c0d18-f067-4fb0-966e-c042151adbb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.780370 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d84d708048a43b6ba3ac01d04416158', 'request_id': 'req-e30c0d18-f067-4fb0-966e-c042151adbb6'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.792771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] The token cache is being invalidate because project 1d84d708048a43b6ba3ac01d04416158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:01.793065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.793726 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-e30c0d18-f067-4fb0-966e-c042151adbb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.795571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1311/2609] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:01 2022] DELETE /identity/v3/projects/1d84d708048a43b6ba3ac01d04416158 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:01.800481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.800929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.801028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.822610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:01.824846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:01.900745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1299/2610] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:01.910504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.911058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.911315 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.936438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:01.937518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:01.945872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1312/2611] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:01.950614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.960801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f431b9-d913-4ded-b01a-9079ca9f737f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.965740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.966401 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.966724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.967002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.967701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:01.968002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.969609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.982739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1300/2612] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:24:01.988125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.998390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7504c778-063a-4cbc-91cf-7910546bd355 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.003949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.004570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.004871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.005138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.005799 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.007708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.036536 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1313/2613] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:01 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:02.040566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.051336 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466e6392-f3d5-4412-b3a4-fb00dbebfb9a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.073936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:02.112955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.113565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.113854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.114120 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.114795 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.116673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.139289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1301/2614] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:02.143378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.154441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24eb27-3416-4cbc-ae20-062a5f2cd5b3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.159558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.160194 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.160448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.160731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.161376 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.163052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.175299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1314/2615] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:02.179370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.191408 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97292a55-d9ca-4357-9bb5-1fedc2c0a888 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.198131 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.198842 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.199117 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.199387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.200347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.201997 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ebcda-8bb5-40c8-8434-4f401b55d013 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.213191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1302/2616] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:02.225087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.239709 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6922a6-f010-4878-80e3-8173e85888cc admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.245049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.245703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.245985 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.246258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] PATH_INFO: `/v3/projects/8a7e568bcbd64a3494c6f23160b8b00c/users/8447eaa5d25e411cb82212ee368f7343/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.262161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a7e568bcbd64a3494c6f23160b8b00c, user_id=8447eaa5d25e411cb82212ee368f7343, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.264848 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.279823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1315/2617] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:02 2022] PUT /identity/v3/projects/8a7e568bcbd64a3494c6f23160b8b00c/users/8447eaa5d25e411cb82212ee368f7343/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:02.290933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.291209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.291484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.326432 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] MFA Rules not processed for user `8447eaa5d25e411cb82212ee368f7343`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:02.327726 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:02.397523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1303/2618] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:02.402353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.471387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a542c-02c0-464b-b82b-01332fd58c9f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.487078 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.487853 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.488222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.488601 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.489327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.491684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.527157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1316/2619] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:02.531576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.543492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a49373-d6af-4f96-bd3d-32290c02f2e8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.629062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.629684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.629966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.630245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.630919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.632910 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.659981 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1304/2620] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:02.664127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.676807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cce25-2d4e-45fd-a379-e8e50a06858b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.682375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.683030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.683441 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.683954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.684767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.686925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.701704 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1317/2621] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:02.708481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.726434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e137ceb1-ba5e-45e3-a0bd-60cf859b14ea admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.734818 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.735637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.735920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.736193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] PATH_INFO: `/v3/projects/a9469daf31e442958553c35a27ca6454/users/986614ef8c4b430aa640d67b676b2d7d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.763549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9469daf31e442958553c35a27ca6454, user_id=986614ef8c4b430aa640d67b676b2d7d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.766609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.777102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:02.785275 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1305/2622] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:02 2022] PUT /identity/v3/projects/a9469daf31e442958553c35a27ca6454/users/986614ef8c4b430aa640d67b676b2d7d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:02.791977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.810432 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44493974-f70f-436b-802f-78d5cbd4b24d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.839783 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:02.911647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.912585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.913045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.913329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.914328 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.917031 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd2922c-b513-4205-bcc6-4e47b96e90f1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.935698 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1318/2623] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:02.942873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.971801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf73c0e-0787-475a-8920-41588c7bd57d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.976821 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.977536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.977817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.978092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] PATH_INFO: `/v3/projects/a9469daf31e442958553c35a27ca6454/users/986614ef8c4b430aa640d67b676b2d7d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.980342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9469daf31e442958553c35a27ca6454, user_id=986614ef8c4b430aa640d67b676b2d7d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.983890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.997311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1306/2624] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:02 2022] PUT /identity/v3/projects/a9469daf31e442958553c35a27ca6454/users/986614ef8c4b430aa640d67b676b2d7d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:03.009847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.010187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.010514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.034031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.035972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.047632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] MFA Rules not processed for user `986614ef8c4b430aa640d67b676b2d7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:03.048964 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.113685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2925c-8cef-4d52-9987-92eb2d6780d5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.118621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.119315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.119656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.120022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] PATH_INFO: `/v3/users/458b41cfb34548a082c1ab01415e8502` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.121452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458b41cfb34548a082c1ab01415e8502)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.123856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.126512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1319/2625] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:03.138865 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.140248 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.214873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8459557e-9e7d-4f7f-b0e1-a5247d776fea None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.219933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.220652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.220937 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.221222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.225069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458b41cfb34548a082c1ab01415e8502'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:03.232947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.234564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81d0d2-b4b7-4889-a8d7-7a54449c8773 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.241658 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458b41cfb34548a082c1ab01415e8502'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:03.246979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1320/2626] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:03.252433 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458b41cfb34548a082c1ab01415e8502'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:03.253186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458b41cfb34548a082c1ab01415e8502'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:03.253847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458b41cfb34548a082c1ab01415e8502'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:03.286749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1307/2627] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:03 2022] DELETE /identity/v3/users/458b41cfb34548a082c1ab01415e8502 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:03.288974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.289424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.289803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.323834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:03.325996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.374129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.375807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.387320 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:03.410069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1321/2628] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:03.423387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.433354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.447595 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4860f-99d9-40f9-b5cb-3220e04546b6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.451733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.452345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.452627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.452911 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.454771 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.477256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:03.507969 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428a6b9-99c6-4ecf-a8c5-130941ad43a0 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.511845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.512494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.512770 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.513052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.523568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.525558 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.535113 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.536717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.537318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1322/2629] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:03.547565 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1308/2630] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:03.911974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.913920 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.931874 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c42c11-b826-4c41-8c52-60003695f591 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.937608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.938201 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.938707 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.938871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] PATH_INFO: `/v3/projects/0b6a13747bea4ea9aba61b67c2715b8c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.940151 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6a13747bea4ea9aba61b67c2715b8c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.942217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.003418 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6a13747bea4ea9aba61b67c2715b8c', 'request_id': 'req-6204c87c-c298-45b3-a2a6-01abcff238ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.009950 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6a13747bea4ea9aba61b67c2715b8c', 'request_id': 'req-6204c87c-c298-45b3-a2a6-01abcff238ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.020426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6a13747bea4ea9aba61b67c2715b8c', 'request_id': 'req-6204c87c-c298-45b3-a2a6-01abcff238ac'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.039998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] The token cache is being invalidate because project 0b6a13747bea4ea9aba61b67c2715b8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:04.040716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.041712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.043451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1323/2631] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:03 2022] DELETE /identity/v3/projects/0b6a13747bea4ea9aba61b67c2715b8c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:04.047828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.059177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73589b3f-1f0a-4748-b6a8-b49e6d87d78f service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.120407 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.120953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.121231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.121502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] PATH_INFO: `/v3/users/32e8775b67484e0096f01a7fbfc9c21f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.122670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32e8775b67484e0096f01a7fbfc9c21f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.124590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.183371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32e8775b67484e0096f01a7fbfc9c21f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.192331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32e8775b67484e0096f01a7fbfc9c21f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.201724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32e8775b67484e0096f01a7fbfc9c21f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.202405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32e8775b67484e0096f01a7fbfc9c21f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.203260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32e8775b67484e0096f01a7fbfc9c21f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.224653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1309/2632] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:04 2022] DELETE /identity/v3/users/32e8775b67484e0096f01a7fbfc9c21f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:04.527991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.530087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.554319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204c87c-c298-45b3-a2a6-01abcff238ac admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.630233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.630809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.631088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.631359 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] PATH_INFO: `/v3/projects/524028509158417e96004b8de2da95cf` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.632511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=524028509158417e96004b8de2da95cf)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.635162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.647555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:04.706475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524028509158417e96004b8de2da95cf', 'request_id': 'req-82dd50d4-2561-41e1-81fe-483594670de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.714867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524028509158417e96004b8de2da95cf', 'request_id': 'req-82dd50d4-2561-41e1-81fe-483594670de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.725305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524028509158417e96004b8de2da95cf', 'request_id': 'req-82dd50d4-2561-41e1-81fe-483594670de3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.737388 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] The token cache is being invalidate because project 524028509158417e96004b8de2da95cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:04.737694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.738471 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.742854 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1324/2633] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:04 2022] DELETE /identity/v3/projects/524028509158417e96004b8de2da95cf => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:04.747576 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.749107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.770313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3e5a3-d09a-438d-bc0f-6e0e412c3e25 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.816702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:04.854503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.855240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.855538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.855823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] PATH_INFO: `/v3/users/547210fc602945cebe18f8b1d31d74a5` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.857051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=547210fc602945cebe18f8b1d31d74a5)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.859164 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.930438 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547210fc602945cebe18f8b1d31d74a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.940082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547210fc602945cebe18f8b1d31d74a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.949222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547210fc602945cebe18f8b1d31d74a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.950449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547210fc602945cebe18f8b1d31d74a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.951335 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-c8c9dd87-1b7c-43d8-ac61-0bd9a57ec7fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547210fc602945cebe18f8b1d31d74a5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.973709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1310/2634] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:04 2022] DELETE /identity/v3/users/547210fc602945cebe18f8b1d31d74a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:05.348734 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.350298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:05.366007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd50d4-2561-41e1-81fe-483594670de3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.440075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.440847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.441168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.441524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] PATH_INFO: `/v3/projects/c6f8f0b48e72462fa61a97772669b7c8` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.442911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f8f0b48e72462fa61a97772669b7c8)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.445152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.511842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f8f0b48e72462fa61a97772669b7c8', 'request_id': 'req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.518677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f8f0b48e72462fa61a97772669b7c8', 'request_id': 'req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.527500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f8f0b48e72462fa61a97772669b7c8', 'request_id': 'req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.541322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] The token cache is being invalidate because project c6f8f0b48e72462fa61a97772669b7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:05.541629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.542422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-40e3e4d2-3ee6-4bdd-aa5c-7f2c731d3c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.544656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1325/2635] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:05 2022] DELETE /identity/v3/projects/c6f8f0b48e72462fa61a97772669b7c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:05.549733 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.550465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.550465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.572810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:05.574181 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:05.656390 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1311/2636] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:05.672772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.673096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.676060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.698398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:05.699678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:05.708397 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1326/2637] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:05.712437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.722832 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d57b8-72c4-4a9d-a39a-0032a45f6112 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.727721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.728839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.729121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.729526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.730190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:05.730510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.732175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.746945 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1312/2638] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:24:05.752291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.766213 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0978ca8d-607d-49d5-8dcc-6e4b29888e91 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.776375 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.777306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.777671 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.777966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.778768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.780801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.818811 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1327/2639] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:05 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:05.823451 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.834303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8ce5-e9ff-4a63-af5a-9cdbb9f7c683 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.895346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:05.919251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.920011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.920295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.920566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.921195 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.923136 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.956636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1313/2640] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:05 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:05.961810 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.968079 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:05.974262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557f0be-54e6-40d3-99f2-3ebc443c3204 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.980870 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.981485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.981758 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.982024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.982705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.984297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:06.001760 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1328/2641] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:06.005957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:06.017610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efacb364-5a18-4ec4-a310-df914911e903 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:06.022503 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:06.023128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:06.023406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:06.023744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:06.024555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:06.026295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1b1b5-3ffb-4b86-b6cb-918c377d686b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:06.038347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1314/2642] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:06.043127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:06.053289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c38997-ee6c-4d7c-8981-09ec1c2d2eff admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:06.058552 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:06.059279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:06.059559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:06.059862 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] PATH_INFO: `/v3/projects/4b5056dc2a8244a0bc6d9a72147a0e7f/users/91af9ceecb6f4bdc8411e73b47e91dcd/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:06.077167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5056dc2a8244a0bc6d9a72147a0e7f, user_id=91af9ceecb6f4bdc8411e73b47e91dcd, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:06.079924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:06.096450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1329/2643] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:06 2022] PUT /identity/v3/projects/4b5056dc2a8244a0bc6d9a72147a0e7f/users/91af9ceecb6f4bdc8411e73b47e91dcd/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:06.101603 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7e05-3157-4ea6-a580-f2ac4573bb4e None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:06.101918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7e05-3157-4ea6-a580-f2ac4573bb4e None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:06.102202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb7e05-3157-4ea6-a580-f2ac4573bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:06.123845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4eeb7e05-3157-4ea6-a580-f2ac4573bb4e None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:06.125393 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4eeb7e05-3157-4ea6-a580-f2ac4573bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:06.198494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1315/2644] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:06.210005 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:06.211725 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:06.282276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07646735-f128-42ee-aea9-3ce4684e47f2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:06.286736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:06.287350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:06.287624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:06.287899 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:06.298042 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:06.299514 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:06.310206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1330/2645] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:07.140721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfc4a-ed9d-450f-a592-388883db91db None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.141346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfc4a-ed9d-450f-a592-388883db91db None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.141346 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcfc4a-ed9d-450f-a592-388883db91db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.178714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-aebcfc4a-ed9d-450f-a592-388883db91db None None] MFA Rules not processed for user `91af9ceecb6f4bdc8411e73b47e91dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:07.180102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-aebcfc4a-ed9d-450f-a592-388883db91db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:07.262313 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1316/2646] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:07.273540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:07.275048 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:07.350467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e518bf-6bb3-4742-8375-9290388e1194 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.356090 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.357016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.357447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.357871 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.372464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.374804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.389967 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1331/2647] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:13.024694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:13.025259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:13.025259 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:13.045811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] MFA Rules not processed for user `91af9ceecb6f4bdc8411e73b47e91dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:13.047059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:13.054866 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1317/2648] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:14.692643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.694107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:14.779454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54bdfaa-0542-4948-b633-705cc63cb5d3 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:14.787072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:14.787800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.788082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.788358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.790294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:14.803350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:14.805000 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bc3f5-56e6-4cb3-b0a0-cc11e0e29862 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:14.817585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1332/2649] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:15.468828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:15.470943 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:15.486099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a42be0-664d-4f9d-8f4b-2d4be364623d None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:15.490724 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:15.491352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:15.491632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:15.491904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] PATH_INFO: `/v3/users/8447eaa5d25e411cb82212ee368f7343` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:15.493041 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8447eaa5d25e411cb82212ee368f7343)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:15.495621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:15.546639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:15.552904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8447eaa5d25e411cb82212ee368f7343'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:15.562539 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8447eaa5d25e411cb82212ee368f7343'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:15.569507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8447eaa5d25e411cb82212ee368f7343'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:15.570143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8447eaa5d25e411cb82212ee368f7343'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:15.570776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8447eaa5d25e411cb82212ee368f7343'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:15.588903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1318/2650] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:15 2022] DELETE /identity/v3/users/8447eaa5d25e411cb82212ee368f7343 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:15.593826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:15.594112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:15.594424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:15.601727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:15.617843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:15.619762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:15.689287 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1333/2651] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:15.699623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:15.701299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:15.765008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b061c5-b28e-4a0d-a4c8-ffa1657e2436 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:15.769496 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:15.770349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:15.770732 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:15.771013 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:15.782383 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:15.783855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:15.794894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1319/2652] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:16.151522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:16.153424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:16.171992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3606ae-e8b5-4343-ae38-02c28877332f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:16.177308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:16.177532 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.177757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.177961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] PATH_INFO: `/v3/projects/8a7e568bcbd64a3494c6f23160b8b00c` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.179196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a7e568bcbd64a3494c6f23160b8b00c)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:16.181002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:16.229666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a7e568bcbd64a3494c6f23160b8b00c', 'request_id': 'req-c059dbfa-bd6a-434e-891b-b2aed7e60af3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.235096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a7e568bcbd64a3494c6f23160b8b00c', 'request_id': 'req-c059dbfa-bd6a-434e-891b-b2aed7e60af3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.243577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a7e568bcbd64a3494c6f23160b8b00c', 'request_id': 'req-c059dbfa-bd6a-434e-891b-b2aed7e60af3'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.255624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] The token cache is being invalidate because project 8a7e568bcbd64a3494c6f23160b8b00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:16.255911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.256625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.258125 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1334/2653] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:16 2022] DELETE /identity/v3/projects/8a7e568bcbd64a3494c6f23160b8b00c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:16.262868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:16.273212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282cbd0-c907-4550-a163-ea439a4a4ce1 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:16.338962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:16.339602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.339874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.340145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] PATH_INFO: `/v3/users/986614ef8c4b430aa640d67b676b2d7d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.341282 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] RBAC: Authorizing `identity:delete_user(user_id=986614ef8c4b430aa640d67b676b2d7d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:16.343228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:16.391969 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '986614ef8c4b430aa640d67b676b2d7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.398772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '986614ef8c4b430aa640d67b676b2d7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.407055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '986614ef8c4b430aa640d67b676b2d7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.408079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '986614ef8c4b430aa640d67b676b2d7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.408679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f6d2f02c-13ea-40c4-8ffe-ba91970b396d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '986614ef8c4b430aa640d67b676b2d7d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.410062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:16.435646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1320/2654] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:16 2022] DELETE /identity/v3/users/986614ef8c4b430aa640d67b676b2d7d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:16.729530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:16.731844 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:16.757817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c059dbfa-bd6a-434e-891b-b2aed7e60af3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:16.828302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:16.828947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.829231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.829538 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] PATH_INFO: `/v3/projects/a9469daf31e442958553c35a27ca6454` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.830724 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9469daf31e442958553c35a27ca6454)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:16.832679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:16.877492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:16.900104 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9469daf31e442958553c35a27ca6454', 'request_id': 'req-29994e7a-ebce-4f85-8135-e1d211f78e51'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.906755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9469daf31e442958553c35a27ca6454', 'request_id': 'req-29994e7a-ebce-4f85-8135-e1d211f78e51'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.916649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9469daf31e442958553c35a27ca6454', 'request_id': 'req-29994e7a-ebce-4f85-8135-e1d211f78e51'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.928394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] The token cache is being invalidate because project a9469daf31e442958553c35a27ca6454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:16.928686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.929458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-29994e7a-ebce-4f85-8135-e1d211f78e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:16.931074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1335/2655] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:16 2022] DELETE /identity/v3/projects/a9469daf31e442958553c35a27ca6454 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:16.937414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.937414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.937414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.961178 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:16.962395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:17.030657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1321/2656] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:17.038627 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.038918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.039199 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.062028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:17.063264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:17.071638 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1336/2657] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:17.075855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.085850 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb17bfc-00a2-43e0-b0b8-28ac710e79cd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.090652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.091265 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.091545 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.091819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.092473 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:17.092796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.094368 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.107803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1322/2658] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:24:17.114506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.123951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bdfe5-d495-4707-8ec0-282fc7db6392 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.129030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.129636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.132753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.132753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.132753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.132753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.164991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1337/2659] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:17 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:17.167000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.177763 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e9766-2f06-47bb-b94c-8b12cc6f304b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.255278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.256031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.256316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.256586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.257338 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.259479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.290021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1323/2660] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:17 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:17.294847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.306246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f629420-c5cb-4314-830d-3a5b0034467c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.310952 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.311564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.311835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.312108 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.314207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.319489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.466541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1338/2661] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:17.470741 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.480889 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f820-0400-4817-9a5b-d08d767459c4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.485691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.486318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.486623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.486900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.487540 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.489188 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422c648e-ca49-4d91-bb2f-fc8a61947106 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.501062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1324/2662] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:17.505181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.516233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4361c3c0-0736-4c8f-97bf-d1b5a700c7c5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.521033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.521684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.521963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.522235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] PATH_INFO: `/v3/projects/50beb48049264dfe8b6a75937c1041fe/users/3de54cdce7634d7d9519722cfb67d45e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.537302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50beb48049264dfe8b6a75937c1041fe, user_id=3de54cdce7634d7d9519722cfb67d45e, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.539968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.553763 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1339/2663] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:17 2022] PUT /identity/v3/projects/50beb48049264dfe8b6a75937c1041fe/users/3de54cdce7634d7d9519722cfb67d45e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:17.559517 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015091-3dc8-434b-aeb5-6e4ebdf191a5 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.559807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015091-3dc8-434b-aeb5-6e4ebdf191a5 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.560081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015091-3dc8-434b-aeb5-6e4ebdf191a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.579721 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-c5015091-3dc8-434b-aeb5-6e4ebdf191a5 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:17.581059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-c5015091-3dc8-434b-aeb5-6e4ebdf191a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:17.623049 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:17.642791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1325/2664] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:17.653988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:17.655804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:17.716504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f77794-eb11-4491-a59e-3df35448ae29 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:17.720337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:17.720880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:17.721156 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:17.721424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:17.733084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:17.734505 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:17.745121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1340/2665] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:18.467622 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752089-51d0-4a61-b499-60db1e7a0cd3 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:18.468375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752089-51d0-4a61-b499-60db1e7a0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:18.468375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9752089-51d0-4a61-b499-60db1e7a0cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:18.499400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d9752089-51d0-4a61-b499-60db1e7a0cd3 None None] MFA Rules not processed for user `3de54cdce7634d7d9519722cfb67d45e`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:18.500980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d9752089-51d0-4a61-b499-60db1e7a0cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:18.589006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1326/2666] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:18.600951 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:18.603957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:18.665777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17036d37-2668-4e02-ac0e-8215e2475c3c service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:18.671788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:18.672400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:18.672680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:18.672948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:18.682231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:18.684082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:18.688801 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:18.697058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1341/2667] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:20.106118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:20.106679 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:20.106802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:20.128104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] MFA Rules not processed for user `3de54cdce7634d7d9519722cfb67d45e`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:20.129330 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:20.136325 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1327/2668] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:20.729546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:20.731507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:20.745403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f8626-bee8-4e25-839e-0a69b9129750 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:20.749222 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:20.749933 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:20.750234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:20.750598 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:20.752475 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:20.766034 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:20.767666 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:20.783271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:20.784278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1342/2669] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:20.786387 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:20.885890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8a16e-5097-4054-bd11-312d23e85244 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:20.892271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:20.893108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:20.893506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:20.893954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:20.896391 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:20.915789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:20.918636 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:20.930490 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1328/2670] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:25.569661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:25.589670 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2dc9f-3535-41c4-a4e0-c68a537d13dd service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:25.596409 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:25.597032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:25.597304 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:25.597568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:25.599467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:25.614377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:25.617772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4114550f-5653-4b88-82f1-56ce24302910 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:25.631561 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1343/2671] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:26.101291 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:26.103051 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:26.119331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc43a43-b25f-4698-ad14-96a83a72fd04 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:26.125522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:26.126125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:26.126399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:26.126864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:26.128843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:26.144214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:26.146031 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:26.158038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1329/2672] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:34.692141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:34.692728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:34.692728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:34.712675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] MFA Rules not processed for user `91af9ceecb6f4bdc8411e73b47e91dcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:34.713878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:34.770055 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1344/2673] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:24:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:36.153435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=81 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:44.763065 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=38 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:49.752712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:49.762510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84087a22-583e-4aa1-8227-aad69caabb97 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:49.766761 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:49.767479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:49.767852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:49.768328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:49.770208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:49.780563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:49.782081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9032a-071c-4ea8-b901-6c0ce695eca0 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:49.790380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1330/2674] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:49.898602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:49.911313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb76dad-ee72-4612-b631-34007d70aa7d None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:49.915465 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:49.916239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:49.916824 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:49.917286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:49.919358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:49.935464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:49.937400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:49.951333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1345/2675] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:59.786800 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:59.944750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=6 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:04.429939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.430416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.430535 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.452967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:04.454094 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.526208 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1331/2676] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:04.582889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:04.582889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.664324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc561082-bc1a-4a0d-a8cc-3a24326c63d2 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:04.674098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:04.675283 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.675767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.676509 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.691140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:04.693453 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:04.702881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1346/2677] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:25:04.877406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:04.882889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:04.895517 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c55528-2da7-4e37-9119-1586cd70c0b5 service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:04.898186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b201e09-352f-4ba3-b4e6-49a704b7edc1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:04.900007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:04.901088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.901693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.902166 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.904454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.904600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:04.905337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.905669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.906017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.910613 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.948913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:04.953673 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:04.983230 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1332/2678] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:05.037754 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:05.039444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:05.084600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1347/2679] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:06.582485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.601826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c928d7f-7b40-46b2-b549-7c409d3efd76 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.630036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.632592 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.632885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.633185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.648024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:06.673469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.675787 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.701488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1333/2680] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:13.796714 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:13.800168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:13.828595 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4c1c-b042-40a7-801c-9a4b190d9cad service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:13.836789 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:13.837489 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:13.838459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:13.838768 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] PATH_INFO: `/v3/users/1f000354b524462bb66812138c6e4695` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:13.841534 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f000354b524462bb66812138c6e4695)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:13.845935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:13.925992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f000354b524462bb66812138c6e4695'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:13.937041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:13.942094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f000354b524462bb66812138c6e4695'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:13.961943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f000354b524462bb66812138c6e4695'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:13.961943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f000354b524462bb66812138c6e4695'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:13.961943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-5fb20f87-66de-452f-b645-b62f6e25ea75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f000354b524462bb66812138c6e4695'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.009620 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1348/2681] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:13 2022] DELETE /identity/v3/users/1f000354b524462bb66812138c6e4695 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:14.346847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:14.360510 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d45f4-694c-42f3-aff0-6e31c903973d service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:14.521987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:14.523331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:14.524456 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:14.524854 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] PATH_INFO: `/v3/projects/0a19aabba33c4e0ab7b3f2d9d58ec621` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:14.526668 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a19aabba33c4e0ab7b3f2d9d58ec621)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:14.529638 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:14.622949 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a19aabba33c4e0ab7b3f2d9d58ec621', 'request_id': 'req-ac6c343c-9d79-4d5c-969f-b42601df66d5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.631334 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a19aabba33c4e0ab7b3f2d9d58ec621', 'request_id': 'req-ac6c343c-9d79-4d5c-969f-b42601df66d5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.649542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a19aabba33c4e0ab7b3f2d9d58ec621', 'request_id': 'req-ac6c343c-9d79-4d5c-969f-b42601df66d5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.681988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] The token cache is being invalidate because project 0a19aabba33c4e0ab7b3f2d9d58ec621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:14.682386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.683249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-ac6c343c-9d79-4d5c-969f-b42601df66d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:14.684936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1334/2682] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:14 2022] DELETE /identity/v3/projects/0a19aabba33c4e0ab7b3f2d9d58ec621 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:14.690703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:14.691001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:14.691286 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:14.733350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:14.737942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:14.898656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1349/2683] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:14.907104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:14.907583 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:14.908052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:14.967670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:14.969084 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:14.979348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1335/2684] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:14.982634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:14.995987 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391366f0-6691-4238-bba7-812f0736f8dd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.006809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.008360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.008887 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.009357 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.011017 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:15.012128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.019373 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.037128 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1350/2685] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:15.043534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.048111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:15.065350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aaeac1-fc1f-4828-8b89-5534c732271a None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.086499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.087237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.087600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.088007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.088820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.092348 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.158722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1336/2686] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:15 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:15.159472 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.179270 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ee45d4-5108-4556-ba6a-c9beafe4c91b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.274083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.275334 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.275689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.276054 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.276864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.279246 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.314775 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1351/2687] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:15 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:15.319845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.343793 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495b06c-8ce9-41ee-ae0b-3ef89a3ad108 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.361899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.361899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.361899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.361899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.362974 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.365382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.385874 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1337/2688] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:15.388562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.400006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a6080-d550-422b-800f-33fadf5a58df admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.405842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.407450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.407857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.408233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.409731 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.412310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8006b-0589-484e-85d7-d9fdfd57e0e1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.425181 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1352/2689] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:15.429532 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.442715 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0c93e-a131-422d-a2c3-0d4538e2d8f7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.463375 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.465138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.465522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.467225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] PATH_INFO: `/v3/projects/279b499cb7b3458c84143ab7f18cab56/users/8b3a6f2d9d3047089863dfa5570f1427/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.489406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] RBAC: Authorizing `identity:create_grant(project_id=279b499cb7b3458c84143ab7f18cab56, user_id=8b3a6f2d9d3047089863dfa5570f1427, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.492522 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.519598 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1338/2690] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:15 2022] PUT /identity/v3/projects/279b499cb7b3458c84143ab7f18cab56/users/8b3a6f2d9d3047089863dfa5570f1427/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:15.532241 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.532840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.533299 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.586235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] MFA Rules not processed for user `8b3a6f2d9d3047089863dfa5570f1427`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:15.587547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:15.649302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:15.748868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1353/2691] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:15.757307 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:15.761796 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:15.898447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9037b8e-7388-4ed4-b52d-4c6aa17a657d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:15.905746 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:15.906078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:15.906299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:15.906606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:15.916229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:15.919378 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378916b6-3719-473d-bb27-fa10a2b7bbb6 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:15.930939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1339/2692] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:16.219926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:16.356374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60cdf-ebdd-4fbf-9373-d5869338ceef None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:16.390876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:16.391898 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:16.392544 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:16.393105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] PATH_INFO: `/v3/users/8b3a6f2d9d3047089863dfa5570f1427` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:16.399404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3a6f2d9d3047089863dfa5570f1427)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:16.402567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:16.529659 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3a6f2d9d3047089863dfa5570f1427'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.542847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3a6f2d9d3047089863dfa5570f1427'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.556204 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3a6f2d9d3047089863dfa5570f1427'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.557585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3a6f2d9d3047089863dfa5570f1427'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.558352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3a6f2d9d3047089863dfa5570f1427'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.610986 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1354/2693] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:16 2022] DELETE /identity/v3/users/8b3a6f2d9d3047089863dfa5570f1427 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:16.615740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:16.616698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:16.617362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:16.669739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:16.671908 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:16.792569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1340/2694] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:16.803662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:16.805276 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:16.810081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:16.812819 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:16.828906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb2c88-4666-40ef-9bae-7bea6a8aa4cd None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:16.834217 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:16.835586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:16.835870 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:16.836146 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] PATH_INFO: `/v3/users/7ebc052a3cb6400a961c1b9d9bec83d6` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:16.860185 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ebc052a3cb6400a961c1b9d9bec83d6)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:16.867200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:16.883252 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:16.887277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:16.888523 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d5e624-54bf-4dd6-9925-715de3987164 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:16.896393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:16.897857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:16.898258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:16.898723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:16.914399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:16.917431 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:16.931515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1355/2695] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:16.956632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ebc052a3cb6400a961c1b9d9bec83d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:16.975579 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ebc052a3cb6400a961c1b9d9bec83d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.002808 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ebc052a3cb6400a961c1b9d9bec83d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.003653 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ebc052a3cb6400a961c1b9d9bec83d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.004441 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ebc052a3cb6400a961c1b9d9bec83d6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.036924 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1341/2696] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:16 2022] DELETE /identity/v3/users/7ebc052a3cb6400a961c1b9d9bec83d6 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:17.360663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:17.379184 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cd721-c1e7-4116-8881-5c3b3fb65df7 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:17.435835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:17.437868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:17.522791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:17.523881 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:17.524265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:17.524800 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] PATH_INFO: `/v3/projects/79389800e0ea49d3b5abfbefe17960ac` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:17.539918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] RBAC: Authorizing `identity:delete_project(project_id=79389800e0ea49d3b5abfbefe17960ac)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:17.542076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:17.610879 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88aba36-7c8d-4612-babd-919ae763a351 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:17.642813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79389800e0ea49d3b5abfbefe17960ac', 'request_id': 'req-99270d36-99ab-4f32-98d7-5dccd4f0025b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.668761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79389800e0ea49d3b5abfbefe17960ac', 'request_id': 'req-99270d36-99ab-4f32-98d7-5dccd4f0025b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.698368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79389800e0ea49d3b5abfbefe17960ac', 'request_id': 'req-99270d36-99ab-4f32-98d7-5dccd4f0025b'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.725656 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] The token cache is being invalidate because project 79389800e0ea49d3b5abfbefe17960ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:17.725915 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.729852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:17.730764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:17.731151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:17.731485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] PATH_INFO: `/v3/projects/279b499cb7b3458c84143ab7f18cab56` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:17.733252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=279b499cb7b3458c84143ab7f18cab56)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:17.733873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.738614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1356/2697] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:17 2022] DELETE /identity/v3/projects/79389800e0ea49d3b5abfbefe17960ac => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:17.738784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:17.739687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:17.741123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:17.773564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99270d36-99ab-4f32-98d7-5dccd4f0025b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:17.822662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:17.824646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:17.826749 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:17.827206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] PATH_INFO: `/v3/users/007a5c65ed4a48648b55d27c6ff12421` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:17.838227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:17.853362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '279b499cb7b3458c84143ab7f18cab56', 'request_id': 'req-fe8c6322-7f8f-4d28-be2e-5d9183025e84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.863754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '279b499cb7b3458c84143ab7f18cab56', 'request_id': 'req-fe8c6322-7f8f-4d28-be2e-5d9183025e84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.884134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '279b499cb7b3458c84143ab7f18cab56', 'request_id': 'req-fe8c6322-7f8f-4d28-be2e-5d9183025e84'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.889515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=007a5c65ed4a48648b55d27c6ff12421)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:17.899526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:17.924218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] The token cache is being invalidate because project 279b499cb7b3458c84143ab7f18cab56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:17.924689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.926009 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-fe8c6322-7f8f-4d28-be2e-5d9183025e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:17.928300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1342/2698] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:17 2022] DELETE /identity/v3/projects/279b499cb7b3458c84143ab7f18cab56 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:17.935784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:17.936070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:17.936350 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:17.995363 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:17.999892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:18.023016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '007a5c65ed4a48648b55d27c6ff12421'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.043467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:18.050134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '007a5c65ed4a48648b55d27c6ff12421'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.069298 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '007a5c65ed4a48648b55d27c6ff12421'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.070691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '007a5c65ed4a48648b55d27c6ff12421'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.071185 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-250014e6-006e-4e20-8e64-d80030022112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '007a5c65ed4a48648b55d27c6ff12421'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.103841 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1357/2699] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:17 2022] DELETE /identity/v3/users/007a5c65ed4a48648b55d27c6ff12421 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:18.142374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1343/2700] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:18.150585 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.151383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.151856 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.177790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:18.179717 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:18.268524 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1358/2701] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:18.280876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.299987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125e70b-29ac-4bd7-9912-bbbfd99bd188 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.311050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.311893 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.312352 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.312806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.313770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:18.314139 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.317639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.350689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1344/2702] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:18.357663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.358927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.368162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf5277-1e0b-4b98-9942-060cc1340d8a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.368647 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:18.375115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.376084 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.376550 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.377022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.377814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.380012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.383826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a7ea-841f-40b8-8fa3-a3af6d818938 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.389240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.389844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.390219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.390502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] PATH_INFO: `/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.407799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d1b7764ba1243a8ae79974d4fa0da01)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.410395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.422641 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1359/2703] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:18 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:18.423624 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.486931 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d1b7764ba1243a8ae79974d4fa0da01', 'request_id': 'req-f79478f4-b9fb-4284-a7f1-320c66dfad68'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.495449 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d1b7764ba1243a8ae79974d4fa0da01', 'request_id': 'req-f79478f4-b9fb-4284-a7f1-320c66dfad68'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.506385 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d1b7764ba1243a8ae79974d4fa0da01', 'request_id': 'req-f79478f4-b9fb-4284-a7f1-320c66dfad68'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.524497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] The token cache is being invalidate because project 3d1b7764ba1243a8ae79974d4fa0da01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:18.524841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.525654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:18.538600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1345/2704] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:18 2022] DELETE /identity/v3/projects/3d1b7764ba1243a8ae79974d4fa0da01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:18.574809 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4bfea-137b-45b1-98f9-b84efdb51be2 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.655588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:18.667280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.669784 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.670631 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.671051 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.672592 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.676852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.714847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1360/2705] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:18 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:18.716461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.718396 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:18.733631 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79478f4-b9fb-4284-a7f1-320c66dfad68 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.740144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.741264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.741812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.742320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.743337 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.745834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.763297 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1346/2706] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:18.764060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.777926 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f5750-2907-4d15-bce1-ad24df9e6ecf admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.783308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.784083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.784372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.784646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.785481 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.787160 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4d3b8-b53f-4b5e-83a8-6f15c738357c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.811593 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1361/2707] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:18.816234 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:18.830205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4b8ba0-c541-4a3a-9f3f-8d1373a04ed1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:18.835212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:18.836237 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.836635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.837038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] PATH_INFO: `/v3/projects/4de82d2024d34e09b359ffe2a4dcfd89/users/079120dd87ed4258b53ce7ce88fa24e6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.853903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de82d2024d34e09b359ffe2a4dcfd89, user_id=079120dd87ed4258b53ce7ce88fa24e6, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:18.857662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:18.866163 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:18.881116 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1347/2708] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:18 2022] PUT /identity/v3/projects/4de82d2024d34e09b359ffe2a4dcfd89/users/079120dd87ed4258b53ce7ce88fa24e6/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:18.892640 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:18.893003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:18.893454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:18.926372 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] MFA Rules not processed for user `079120dd87ed4258b53ce7ce88fa24e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:18.927840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:19.022663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1362/2709] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:19.025472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.133947 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857576-5371-4fca-b99d-5fd52d40744f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.140512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.141244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.141649 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.141964 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.142776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.144812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.205260 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1348/2710] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:19 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:19.209302 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.221994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbe44e-b55e-4cfc-8883-be441b541e7a None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.347439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.348371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.348816 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.349236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.350032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.358047 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.395325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1363/2711] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:19 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:19.399440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.413022 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663acc5-8c72-4711-8579-da87943ec6cf admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.418979 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.419817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.420101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.420377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.421044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.423244 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.440086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1349/2712] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:19.444289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.458633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe89f4e-ac69-4a8a-8745-088becf60f99 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.469924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.470828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.471206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.471597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] PATH_INFO: `/v3/projects/4d6758640379440587cfaf70c99b1c40/users/9fc6639fa3df45859a8a244e0302ddfa/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.492645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6758640379440587cfaf70c99b1c40, user_id=9fc6639fa3df45859a8a244e0302ddfa, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.496023 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.516712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1364/2713] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:19 2022] PUT /identity/v3/projects/4d6758640379440587cfaf70c99b1c40/users/9fc6639fa3df45859a8a244e0302ddfa/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:19.520928 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.532134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fae32d-5e90-4024-bfb5-966cf8bd3790 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.611261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.612059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.612499 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.612805 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.613571 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.615332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5683c6a7-f28f-4632-90a7-44e9d588fd86 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.627617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1350/2714] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:19.631835 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.646239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a295e27-71ab-407e-a876-5eb8400a29e3 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.652223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.653039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.654257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.654932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] PATH_INFO: `/v3/projects/4d6758640379440587cfaf70c99b1c40/users/9fc6639fa3df45859a8a244e0302ddfa/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.657382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d6758640379440587cfaf70c99b1c40, user_id=9fc6639fa3df45859a8a244e0302ddfa, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.660179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.674826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1365/2715] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:19 2022] PUT /identity/v3/projects/4d6758640379440587cfaf70c99b1c40/users/9fc6639fa3df45859a8a244e0302ddfa/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:19.687611 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.688069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.688311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.721300 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] MFA Rules not processed for user `9fc6639fa3df45859a8a244e0302ddfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:19.722553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:19.801657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:19.828809 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1351/2716] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:19.843308 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:19.845123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:19.882041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:19.943745 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22123621-7694-4ba6-b80a-ec0e9c6727e5 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:19.948124 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:19.948927 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:19.949414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:19.949849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:19.963136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:19.964700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:19.977826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1366/2717] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:19 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:20.552448 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:20.562812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34875c-eea8-4773-8ed6-ec240467ee6b None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:20.568105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc6639fa3df45859a8a244e0302ddfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d6758640379440587cfaf70c99b1c40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:20.569101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:20.569521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:20.569906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:20.570772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:20.571251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:20.573897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:20.591098 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1352/2718] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:25:20.595484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:20.613178 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957bc0-26f0-4289-92e7-82b55497a013 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:20.621868 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc6639fa3df45859a8a244e0302ddfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d6758640379440587cfaf70c99b1c40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:20.625324 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:20.626484 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:20.627264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:20.628545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:20.633936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:20.669107 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1367/2719] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:20 2022] POST /identity/v3/projects => generated 354 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:20.674354 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:20.699862 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de493c-2563-48f8-9312-ffc2168afca9 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:20.780554 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc6639fa3df45859a8a244e0302ddfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d6758640379440587cfaf70c99b1c40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:20.781156 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:20.781434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:20.781975 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:20.782725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:20.783431 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:20.785585 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:20.802594 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1353/2720] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:20.860476 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:20.883319 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b17335-0f16-499c-831a-ef0aa8f94f01 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:20.888840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9fc6639fa3df45859a8a244e0302ddfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4d6758640379440587cfaf70c99b1c40', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:20.890618 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:20.891080 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:20.891636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] PATH_INFO: `/v3/projects/271a9fc2587e4dc18d421eae07c1c470` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:20.894893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorizing `identity:delete_project(project_id=271a9fc2587e4dc18d421eae07c1c470)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:20.897622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:20.963929 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271a9fc2587e4dc18d421eae07c1c470', 'request_id': 'req-f930dbd6-73f5-4042-862e-d5161a87ea5d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:20.976645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271a9fc2587e4dc18d421eae07c1c470', 'request_id': 'req-f930dbd6-73f5-4042-862e-d5161a87ea5d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:20.988683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271a9fc2587e4dc18d421eae07c1c470', 'request_id': 'req-f930dbd6-73f5-4042-862e-d5161a87ea5d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.005313 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] The token cache is being invalidate because project 271a9fc2587e4dc18d421eae07c1c470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:21.005597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.006329 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f930dbd6-73f5-4042-862e-d5161a87ea5d tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.008152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1368/2721] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:20 2022] DELETE /identity/v3/projects/271a9fc2587e4dc18d421eae07c1c470 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:21.271226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:21.273093 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:21.368916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a90d9c-d89a-4a74-80fb-612ce9b7b181 tempest-VolumeQuotaClassesTest-674629775 tempest-VolumeQuotaClassesTest-674629775-project-admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:21.445355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:21.446196 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:21.446612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:21.447144 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] PATH_INFO: `/v3/users/079120dd87ed4258b53ce7ce88fa24e6` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:21.448988 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=079120dd87ed4258b53ce7ce88fa24e6)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:21.451089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:21.478657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:21.519769 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '079120dd87ed4258b53ce7ce88fa24e6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.529452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '079120dd87ed4258b53ce7ce88fa24e6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.542148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '079120dd87ed4258b53ce7ce88fa24e6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.544065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '079120dd87ed4258b53ce7ce88fa24e6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.545326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '079120dd87ed4258b53ce7ce88fa24e6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:21.592210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1354/2722] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:21 2022] DELETE /identity/v3/users/079120dd87ed4258b53ce7ce88fa24e6 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:21.597705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:21.597705 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:21.599068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:21.621016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:21.622647 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:21.758153 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1369/2723] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:21.758920 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:21.761575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:21.846621 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e40ac2c-931b-4571-b505-5597e3229953 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:21.858755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:21.859700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:21.860000 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:21.860328 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:21.883370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:21.884492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:21.896705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1355/2724] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:22.271041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:22.272935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:22.309564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df916c94-aed5-46d5-a41d-4cd26e6b9d04 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:22.317045 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:22.317925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:22.318530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:22.318953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] PATH_INFO: `/v3/projects/4de82d2024d34e09b359ffe2a4dcfd89` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:22.321422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de82d2024d34e09b359ffe2a4dcfd89)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:22.324983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:22.353448 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:22.404774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de82d2024d34e09b359ffe2a4dcfd89', 'request_id': 'req-983f0628-b155-4102-b48d-a221da5137c1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.413941 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de82d2024d34e09b359ffe2a4dcfd89', 'request_id': 'req-983f0628-b155-4102-b48d-a221da5137c1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.425092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de82d2024d34e09b359ffe2a4dcfd89', 'request_id': 'req-983f0628-b155-4102-b48d-a221da5137c1'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.444081 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] The token cache is being invalidate because project 4de82d2024d34e09b359ffe2a4dcfd89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:22.444625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.445657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.447318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1370/2725] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:22 2022] DELETE /identity/v3/projects/4de82d2024d34e09b359ffe2a4dcfd89 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:22.451596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:22.462534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad6315-c2f1-49fa-b4eb-76e7e5c876db service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:22.539543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:22.540319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:22.540670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:22.541059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] PATH_INFO: `/v3/users/9fc6639fa3df45859a8a244e0302ddfa` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:22.543182 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc6639fa3df45859a8a244e0302ddfa)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:22.545536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:22.632154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:22.638130 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc6639fa3df45859a8a244e0302ddfa'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.649488 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc6639fa3df45859a8a244e0302ddfa'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.661778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc6639fa3df45859a8a244e0302ddfa'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.662896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc6639fa3df45859a8a244e0302ddfa'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.663550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-884eb6d9-2e6b-4fbe-93ff-9f3939a055fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc6639fa3df45859a8a244e0302ddfa'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:22.690706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1356/2726] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:22 2022] DELETE /identity/v3/users/9fc6639fa3df45859a8a244e0302ddfa => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:23.032776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.035380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:23.059684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f0628-b155-4102-b48d-a221da5137c1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.147867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.148911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.149385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.149959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] PATH_INFO: `/v3/projects/4d6758640379440587cfaf70c99b1c40` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.152115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d6758640379440587cfaf70c99b1c40)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.154395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.217223 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d6758640379440587cfaf70c99b1c40', 'request_id': 'req-c2175a11-084e-4871-85da-93021c448a9f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:23.225906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d6758640379440587cfaf70c99b1c40', 'request_id': 'req-c2175a11-084e-4871-85da-93021c448a9f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:23.235429 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d6758640379440587cfaf70c99b1c40', 'request_id': 'req-c2175a11-084e-4871-85da-93021c448a9f'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:23.248401 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] The token cache is being invalidate because project 4d6758640379440587cfaf70c99b1c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:23.248897 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:23.249822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-c2175a11-084e-4871-85da-93021c448a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:23.253189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1371/2727] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:23 2022] DELETE /identity/v3/projects/4d6758640379440587cfaf70c99b1c40 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:23.259147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.260205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.260868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.285497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:23.287430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:23.379415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1357/2728] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:23.384119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.384609 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.385016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.415214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:23.417776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:23.429947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1372/2729] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:23.436719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.449006 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aea3c6-5205-4fa3-a598-027998b0e9da None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.455170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.456358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.456654 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.457190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.458461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:23.459033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.461340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.482829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1358/2730] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:23.485867 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.498400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13193fb3-b86a-4fe8-9908-daabd303adef None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.505792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.506494 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.506956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.507325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.508424 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.512273 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.553710 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1373/2731] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:23 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:23.560784 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.576481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d88fe-eb51-4090-a0f3-efbbcbe08dc7 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.737968 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.738507 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.738830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.739105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.739843 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.742189 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.768977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1359/2732] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:23 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:23.776099 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.788537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2c63d-35aa-4ecb-acb6-5da4c8f646d9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.795010 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.795686 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.795963 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.796234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.796922 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.799096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.801243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:23.832684 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1374/2733] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:23.834954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.870700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6ff3cc-4103-4614-bb67-1239ac0ad08f admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.892406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.893822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.894414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.895463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:23.896544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:23.898637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c446550-5a51-448b-b1a7-e2758cdee7bf admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:23.930474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1360/2734] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:23.932774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:23.953086 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c84cf4-c9a1-47c7-9e99-1fb0af96703f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:23.976909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:23.977884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:23.978461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:23.978863 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] PATH_INFO: `/v3/projects/aa8d8067861c427089162e7106300b50/users/fe95a8ce68e04c83b0cd199f8b539ac3/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.017460 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8d8067861c427089162e7106300b50, user_id=fe95a8ce68e04c83b0cd199f8b539ac3, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.021132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.054652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1375/2735] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:23 2022] PUT /identity/v3/projects/aa8d8067861c427089162e7106300b50/users/fe95a8ce68e04c83b0cd199f8b539ac3/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:24.065791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.066180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.066542 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.119536 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] MFA Rules not processed for user `fe95a8ce68e04c83b0cd199f8b539ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:24.122593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:24.163719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:24.241982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1361/2736] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:24.243752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.345369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7aaf49-1815-4853-9a0d-fd0aefd88d0c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.360110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.361139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.361662 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.362165 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.363225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.367896 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.418530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1376/2737] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:24 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:24.422886 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.436283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5645-e5e3-4837-9be7-9be76da31eeb None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.513982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.514885 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.515168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.515440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.516245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.518310 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.544609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1362/2738] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:24 2022] POST /identity/v3/users => generated 297 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:24.550157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.567877 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe75f81-8652-41f2-a49b-f075aaf1b018 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.578094 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.579063 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.579568 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.579829 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.580369 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.582118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.609327 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1377/2739] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:24.617901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.630624 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20c1f4-1c10-4753-b5d4-ea3973e3ef6c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.638227 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.639830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.640309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.640747 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] PATH_INFO: `/v3/projects/5c09356141834ea2b00ef1b6c9b230f9/users/091efcf6ce414b58978784d06babb67d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.670460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c09356141834ea2b00ef1b6c9b230f9, user_id=091efcf6ce414b58978784d06babb67d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.676772 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.700190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1363/2740] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:24 2022] PUT /identity/v3/projects/5c09356141834ea2b00ef1b6c9b230f9/users/091efcf6ce414b58978784d06babb67d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:24.706365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.722406 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a18e51-42eb-4b04-b0d9-d564a1084f0f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.822607 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.823913 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.824190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.824456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.825111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.829186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4605cbe-e40e-439e-800f-81dfe1591be0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.844855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1378/2741] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:24.846039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:24.863828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159fe2aa-d83d-4200-b12a-47b3f9b8fa4b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:24.871838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:24.872531 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.872812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.873087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] PATH_INFO: `/v3/projects/5c09356141834ea2b00ef1b6c9b230f9/users/091efcf6ce414b58978784d06babb67d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.876212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c09356141834ea2b00ef1b6c9b230f9, user_id=091efcf6ce414b58978784d06babb67d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:24.878748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:24.891657 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1364/2742] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:24 2022] PUT /identity/v3/projects/5c09356141834ea2b00ef1b6c9b230f9/users/091efcf6ce414b58978784d06babb67d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:24.903855 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:24.904158 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:24.904411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:24.937889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] MFA Rules not processed for user `091efcf6ce414b58978784d06babb67d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:24.939202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:24.991571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:25.064050 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1379/2743] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:25.080627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:25.082883 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:25.177148 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abfc9e-ae5b-4511-b24d-39f101f83ca5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:25.184106 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:25.185071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:25.185523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:25.185967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:25.207007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:25.209092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53658d73-dd0f-4cb4-8545-be95d460bccf service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:25.224202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1365/2744] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:25.395265 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:25.405796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0575c20-c67f-4faa-b3cc-c536e1164714 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:25.410500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:25.411269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:25.411583 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:25.411880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:25.497363 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:25.500087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:25.515649 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1380/2745] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:27.896520 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:27.897021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:27.897219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:27.918890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:27.920175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:27.926277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:27.982954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1366/2746] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:27.998860 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:28.000733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:28.085245 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71e91-b454-4d32-823b-ea5e78005552 service cinder] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:28.090981 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:28.091742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:28.092025 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:28.092374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:28.101957 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:28.103608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee46288-c5f7-4345-a14e-1b5c78956ac7 service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:28.113793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1381/2747] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:25:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:25:32.527470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:32.593776 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33212e9-33c8-4337-9894-e1d0592f365f None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:32.598129 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:32.598786 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:32.599065 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:32.599347 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] PATH_INFO: `/v3/users/fe95a8ce68e04c83b0cd199f8b539ac3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:32.600472 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe95a8ce68e04c83b0cd199f8b539ac3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:32.602380 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:32.648757 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe95a8ce68e04c83b0cd199f8b539ac3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:32.655530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe95a8ce68e04c83b0cd199f8b539ac3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:32.663422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe95a8ce68e04c83b0cd199f8b539ac3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:32.664050 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe95a8ce68e04c83b0cd199f8b539ac3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:32.664646 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe95a8ce68e04c83b0cd199f8b539ac3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:32.681894 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1367/2748] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:32 2022] DELETE /identity/v3/users/fe95a8ce68e04c83b0cd199f8b539ac3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:32.687013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:32.687486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:32.687634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:32.706553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:32.707588 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:32.760083 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:32.782032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1382/2749] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:32.795264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:32.796749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:32.851612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:32.882516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc6231a-3405-4a2d-a98b-3a02a9ed5230 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:32.886698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:32.887371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:32.887651 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:32.887930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:32.898114 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:32.899614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:32.910677 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1368/2750] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:33.342468 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:33.344019 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:33.357377 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bc1b4b-be66-48d0-8c1d-945e259dfa5f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:33.362438 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:33.363105 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:33.363383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:33.363654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] PATH_INFO: `/v3/projects/aa8d8067861c427089162e7106300b50` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:33.364803 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8d8067861c427089162e7106300b50)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:33.366750 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:33.420358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:33.422119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:33.435651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8d8067861c427089162e7106300b50', 'request_id': 'req-2c52ac68-029c-4b00-a65d-d9677302268d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.438447 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e294b-4092-4e20-8bf1-9d9e8a5c3ef4 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:33.442353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8d8067861c427089162e7106300b50', 'request_id': 'req-2c52ac68-029c-4b00-a65d-d9677302268d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.451728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8d8067861c427089162e7106300b50', 'request_id': 'req-2c52ac68-029c-4b00-a65d-d9677302268d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.464701 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] The token cache is being invalidate because project aa8d8067861c427089162e7106300b50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:33.465112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.465842 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.467518 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1383/2751] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:33 2022] DELETE /identity/v3/projects/aa8d8067861c427089162e7106300b50 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:33.471526 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:33.472843 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:33.497807 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:33.498896 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:33.499048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:33.499324 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] PATH_INFO: `/v3/users/91af9ceecb6f4bdc8411e73b47e91dcd` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:33.500519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91af9ceecb6f4bdc8411e73b47e91dcd)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:33.502476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:33.545912 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52ac68-029c-4b00-a65d-d9677302268d admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:33.551752 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:33.552504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:33.552786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:33.553230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] PATH_INFO: `/v3/users/091efcf6ce414b58978784d06babb67d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:33.554345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=091efcf6ce414b58978784d06babb67d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:33.556356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:33.557521 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91af9ceecb6f4bdc8411e73b47e91dcd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.566167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91af9ceecb6f4bdc8411e73b47e91dcd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.576882 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91af9ceecb6f4bdc8411e73b47e91dcd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.577500 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91af9ceecb6f4bdc8411e73b47e91dcd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.578174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91af9ceecb6f4bdc8411e73b47e91dcd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.600860 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1369/2752] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:33 2022] DELETE /identity/v3/users/91af9ceecb6f4bdc8411e73b47e91dcd => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:33.611348 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091efcf6ce414b58978784d06babb67d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.624150 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091efcf6ce414b58978784d06babb67d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.632634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091efcf6ce414b58978784d06babb67d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.633452 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091efcf6ce414b58978784d06babb67d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.634244 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091efcf6ce414b58978784d06babb67d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:33.649279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:33.654508 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1384/2753] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:33 2022] DELETE /identity/v3/users/091efcf6ce414b58978784d06babb67d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:33.789561 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:33.791169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:33.888738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:33.893017 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c07f4e-589b-4724-ab92-882386cfea50 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:33.955570 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:33.957126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:33.961516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:33.962153 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:33.962465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:33.962770 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] PATH_INFO: `/v3/projects/4b5056dc2a8244a0bc6d9a72147a0e7f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:33.964212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5056dc2a8244a0bc6d9a72147a0e7f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:33.966212 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:33.973485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee4b30-540f-491f-bbb9-d8010a372bb9 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:33.987597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:33.988292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:33.988577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:33.988884 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] PATH_INFO: `/v3/projects/5c09356141834ea2b00ef1b6c9b230f9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:33.990501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c09356141834ea2b00ef1b6c9b230f9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:33.992610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.018485 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5056dc2a8244a0bc6d9a72147a0e7f', 'request_id': 'req-added8ff-b00c-4431-8597-02f548b15ca5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.025830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5056dc2a8244a0bc6d9a72147a0e7f', 'request_id': 'req-added8ff-b00c-4431-8597-02f548b15ca5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.033341 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5056dc2a8244a0bc6d9a72147a0e7f', 'request_id': 'req-added8ff-b00c-4431-8597-02f548b15ca5'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.046147 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] The token cache is being invalidate because project 4b5056dc2a8244a0bc6d9a72147a0e7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:34.046405 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c09356141834ea2b00ef1b6c9b230f9', 'request_id': 'req-8d798fe4-4f22-44ba-b3fc-163525358457'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.046541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.047174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-added8ff-b00c-4431-8597-02f548b15ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.048699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1370/2754] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:33 2022] DELETE /identity/v3/projects/4b5056dc2a8244a0bc6d9a72147a0e7f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:34.052944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c09356141834ea2b00ef1b6c9b230f9', 'request_id': 'req-8d798fe4-4f22-44ba-b3fc-163525358457'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.054897 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226332-76bf-4285-8cd8-d758d572f023 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.055190 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226332-76bf-4285-8cd8-d758d572f023 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.055468 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b226332-76bf-4285-8cd8-d758d572f023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.061196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c09356141834ea2b00ef1b6c9b230f9', 'request_id': 'req-8d798fe4-4f22-44ba-b3fc-163525358457'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.072597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] The token cache is being invalidate because project 5c09356141834ea2b00ef1b6c9b230f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:34.072924 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.073622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-8d798fe4-4f22-44ba-b3fc-163525358457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:34.075132 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1385/2755] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:33 2022] DELETE /identity/v3/projects/5c09356141834ea2b00ef1b6c9b230f9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:34.078558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6b226332-76bf-4285-8cd8-d758d572f023 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.078558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6b226332-76bf-4285-8cd8-d758d572f023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.080253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce500d8-43f4-4c1b-8d96-14f79c022d6f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.080543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce500d8-43f4-4c1b-8d96-14f79c022d6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.080813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce500d8-43f4-4c1b-8d96-14f79c022d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.100746 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-fce500d8-43f4-4c1b-8d96-14f79c022d6f None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.101943 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-fce500d8-43f4-4c1b-8d96-14f79c022d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.148071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1371/2756] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.150010 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.150365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.151020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.171411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1386/2757] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.173700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.174934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.180127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.180512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.180939 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.183919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1372/2758] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.187919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.197412 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64ebeb-c4d5-4ad4-8a05-2710e87eaf56 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.200918 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.202137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.202361 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.202797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.203089 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.203376 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.204289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80490) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:34.204599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.206271 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.210680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1387/2759] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.215028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.222071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1373/2760] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:25:34.223104 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.224352 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f02b4-0436-4f7c-b441-19b99c33f642 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.228984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.230230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.230444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.230748 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.231394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:34.231694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.233443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.233694 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212370f-1616-47d6-84b9-93f8bbd3fe7d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.239187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.239791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.240072 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.240379 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.241002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.242926 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.245882 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1388/2761] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:25:34.250467 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.259086 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:34.261191 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b5e87-ac45-4b30-b08c-f9ddcb7bfe5b admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.267776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.268487 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.268814 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.269113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.269715 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.271657 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.277530 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1374/2762] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:34.278998 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.303956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1389/2763] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:34.305106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.345074 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ec529-bb1f-49e4-bac1-9b9ccf595d20 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.350521 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:34.371321 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8392c93-2dae-43c7-a1e1-6be97db3c248 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.376486 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.377100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.377527 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.377796 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.378425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.380331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.409946 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1390/2764] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:34.411856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.412529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.412814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.413087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] PATH_INFO: `/v3/users` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.413722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.415604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.416021 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.429606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b2830-dae2-4d80-b55a-36f181af0393 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.436089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.437143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.437545 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.438009 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.438680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.440385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.441591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1375/2765] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:34.446187 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.454954 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1391/2766] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:34.460788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.461199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a1b04-a78d-4774-a3ff-9709766b254b admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.467003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.467634 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.467919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.468209 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.468869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.473617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.475368 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db010e-4250-48ba-8766-6595af3a0def admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.482689 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.483815 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.484253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.484678 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.486029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1376/2767] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:34.486221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.488788 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.489748 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.500025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd95673-6164-402e-91cb-f0856aac2dd1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.507919 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1392/2768] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:34.512291 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.515828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.516864 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.517373 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.517936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.519083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.521836 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.529282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f5a825-928e-49f8-9579-90b5465f599c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.535850 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.536579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.536910 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.537253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] PATH_INFO: `/v3/projects/f8e4ce4eaa8a49d1b875fb5109839744/users/d53b88a0d6ae4b758bb15f4f6322b3b4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.537967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1377/2769] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:34.542767 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.552541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072e08a-2a4d-4dd2-9c0c-d9f57d6be005 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.553687 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8e4ce4eaa8a49d1b875fb5109839744, user_id=d53b88a0d6ae4b758bb15f4f6322b3b4, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.556533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c021a6c7-2ea1-45b0-bd96-3c40a9765f4c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.557177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.557916 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.558477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.558607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] PATH_INFO: `/v3/projects/c56abb478c1f44508d3f22ab53b0af8c/users/de84a91a0f6e4a58813a48a3bc189f9a/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.570333 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1393/2770] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:34 2022] PUT /identity/v3/projects/f8e4ce4eaa8a49d1b875fb5109839744/users/d53b88a0d6ae4b758bb15f4f6322b3b4/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:34.575143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56abb478c1f44508d3f22ab53b0af8c, user_id=de84a91a0f6e4a58813a48a3bc189f9a, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.577828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb1954-995e-4be1-b2dd-58baea110245 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.580636 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.580925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.581197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.593652 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1378/2771] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:34 2022] PUT /identity/v3/projects/c56abb478c1f44508d3f22ab53b0af8c/users/de84a91a0f6e4a58813a48a3bc189f9a/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:34.612234 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] MFA Rules not processed for user `d53b88a0d6ae4b758bb15f4f6322b3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.613490 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.661687 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.661961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.662231 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.685235 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1394/2772] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.689833 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.692264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] MFA Rules not processed for user `de84a91a0f6e4a58813a48a3bc189f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:34.693717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.730566 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:34.769249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a07007-02b0-457f-a437-4c9e37b1ba16 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.776762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.777353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.777634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.777903 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.778553 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.778873 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1379/2773] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:34.780639 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.789345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.790696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:34.802909 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:34.811637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1395/2774] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:34.815576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.848223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac1c5e4-c87a-4628-9740-1a834c338231 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.883155 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207fdfe1-df9c-4e70-b785-cd425f7a5dd0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.888390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.889098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.889384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.889690 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.890480 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.892414 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.903083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.903697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.903980 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.904258 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.914599 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1396/2775] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:34 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:34.919027 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.928902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec83675-9c03-4d52-8150-39b5e9a9c38f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.933652 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.934242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.934520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.934928 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.935440 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.937052 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.947198 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1397/2776] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:34.952014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:34.961786 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ddf64-2e78-4e18-b567-2d76fa69bcd1 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:34.962365 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.964458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:34.966996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:34.967804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:34.968087 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:34.968365 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] PATH_INFO: `/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb/users/dd89607a6a784b8c9f82022640c23b4d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:34.974617 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1380/2777] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:34.983879 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a540e47cc3da4ca7b4feaa42903d68cb, user_id=dd89607a6a784b8c9f82022640c23b4d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:34.986032 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.000572 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1398/2778] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:34 2022] PUT /identity/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb/users/dd89607a6a784b8c9f82022640c23b4d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:35.004623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:35.015957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78708cbe-89dc-4301-9c84-80184466d264 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:35.058847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:35.075749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:35.076353 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.076659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.076934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.077569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:35.079460 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7b95f-a8d9-4a0d-84e3-8cc92902539d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.091035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1381/2779] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:35.095804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:35.106443 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7e0f37-2726-46e3-a3d8-24a1cd53ba9f admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:35.114140 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:35.114947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.115307 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.115692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] PATH_INFO: `/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb/users/dd89607a6a784b8c9f82022640c23b4d/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.118207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a540e47cc3da4ca7b4feaa42903d68cb, user_id=dd89607a6a784b8c9f82022640c23b4d, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:35.121559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.134935 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1399/2780] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:35 2022] PUT /identity/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb/users/dd89607a6a784b8c9f82022640c23b4d/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:35.154177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.154689 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.155151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.197415 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] MFA Rules not processed for user `dd89607a6a784b8c9f82022640c23b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:35.200032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:35.282626 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1382/2781] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:35.294727 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:35.364813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec8cd-495f-4855-bb74-101a404d7613 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:35.368454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:35.369058 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.369337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.369608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.380548 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:35.381992 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e407a460-a043-4857-b83c-bbb989b6e460 service cinder] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.391755 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1400/2782] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:35.523881 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:35.589543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e3bf7-0c56-4639-9be1-34f2af6c06f2 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:35.594444 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:35.595105 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.595388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.595663 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] PATH_INFO: `/v3/users/d53b88a0d6ae4b758bb15f4f6322b3b4` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.596798 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53b88a0d6ae4b758bb15f4f6322b3b4)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:35.598738 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.656162 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53b88a0d6ae4b758bb15f4f6322b3b4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:35.663167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53b88a0d6ae4b758bb15f4f6322b3b4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:35.671857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53b88a0d6ae4b758bb15f4f6322b3b4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:35.672474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53b88a0d6ae4b758bb15f4f6322b3b4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:35.673083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53b88a0d6ae4b758bb15f4f6322b3b4'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:35.677835 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:35.693936 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1383/2783] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:35 2022] DELETE /identity/v3/users/d53b88a0d6ae4b758bb15f4f6322b3b4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:35.700683 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.701127 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.701916 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.724162 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:35.725436 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:35.770325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:35.794293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1401/2784] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:35.803994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:35.805435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:35.871477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7c9c8-e4f3-4d7c-89de-e59efa5a9f65 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:35.875229 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:35.875834 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:35.876140 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:35.876414 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:35.885199 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:35.887205 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:35.896538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1384/2785] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:36.166729 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:36.168122 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:36.180164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abf1825-abec-4eb9-b5fc-37d1c513ba79 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:36.185145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:36.185771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:36.186046 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:36.186318 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] PATH_INFO: `/v3/projects/f8e4ce4eaa8a49d1b875fb5109839744` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:36.187580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8e4ce4eaa8a49d1b875fb5109839744)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:36.189637 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:36.238392 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8e4ce4eaa8a49d1b875fb5109839744', 'request_id': 'req-f163a587-1209-4e00-8dc5-192575721936'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.243785 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8e4ce4eaa8a49d1b875fb5109839744', 'request_id': 'req-f163a587-1209-4e00-8dc5-192575721936'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.251383 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8e4ce4eaa8a49d1b875fb5109839744', 'request_id': 'req-f163a587-1209-4e00-8dc5-192575721936'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.262688 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] The token cache is being invalidate because project f8e4ce4eaa8a49d1b875fb5109839744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:36.262974 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.263622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-f163a587-1209-4e00-8dc5-192575721936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.265212 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1402/2786] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:36 2022] DELETE /identity/v3/projects/f8e4ce4eaa8a49d1b875fb5109839744 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:36.269333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:36.279705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda1cba-8e3d-4bbd-9ca4-dc615b550a8a service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:36.343377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:36.344232 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:36.344543 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:36.345170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] PATH_INFO: `/v3/users/dd89607a6a784b8c9f82022640c23b4d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:36.346034 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd89607a6a784b8c9f82022640c23b4d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:36.347987 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:36.383112 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d905f0-05f0-4629-b1ad-493b32c1f5ec None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:36.383400 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d905f0-05f0-4629-b1ad-493b32c1f5ec None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:36.383680 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d905f0-05f0-4629-b1ad-493b32c1f5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:36.402469 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd89607a6a784b8c9f82022640c23b4d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.403243 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-95d905f0-05f0-4629-b1ad-493b32c1f5ec None None] MFA Rules not processed for user `de84a91a0f6e4a58813a48a3bc189f9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:36.404447 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-95d905f0-05f0-4629-b1ad-493b32c1f5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:36.409838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd89607a6a784b8c9f82022640c23b4d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.418048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd89607a6a784b8c9f82022640c23b4d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.418695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd89607a6a784b8c9f82022640c23b4d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.419377 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd89607a6a784b8c9f82022640c23b4d'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.439599 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1385/2787] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:36 2022] DELETE /identity/v3/users/dd89607a6a784b8c9f82022640c23b4d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:36.463310 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1403/2788] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:36.775697 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:36.777320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:36.787995 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:36.880394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e29e9-fb1b-446a-a7eb-cd9211f68191 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:36.885055 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:36.885722 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:36.886005 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:36.886277 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] PATH_INFO: `/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:36.887674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a540e47cc3da4ca7b4feaa42903d68cb)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:36.889604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:36.949081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a540e47cc3da4ca7b4feaa42903d68cb', 'request_id': 'req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.957167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a540e47cc3da4ca7b4feaa42903d68cb', 'request_id': 'req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.971785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a540e47cc3da4ca7b4feaa42903d68cb', 'request_id': 'req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.989406 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] The token cache is being invalidate because project a540e47cc3da4ca7b4feaa42903d68cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:36.989696 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.990508 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-103cc0cd-c2fe-481a-b494-cd0c9db2ef7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:36.992455 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1386/2789] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:36 2022] DELETE /identity/v3/projects/a540e47cc3da4ca7b4feaa42903d68cb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:36.997211 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:36.997500 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:36.997804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.034994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:37.036306 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:37.108866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1404/2790] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:37.174614 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.174905 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.175180 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.195218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:37.196425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:37.215210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1387/2791] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:37.220169 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.230413 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72322480-71d3-466b-9cdf-7eb4e7e1b4bb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.235217 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.235808 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.236088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.236399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.237297 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:37.237632 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.239537 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.248253 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:37.255606 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1405/2792] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:37.266370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.275871 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc349-dbc5-481d-81ec-03101a63b184 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.280921 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.281605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.281801 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.282078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.282863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.284803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.319965 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1388/2793] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:37 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:37.324419 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.335461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8be02-c421-4f42-84a3-8f4590dc668e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.421039 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.423530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.423530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.423530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.423530 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.425822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.450861 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1406/2794] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:37 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:25:37.455251 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.465914 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1c08c-bd45-475b-a2fb-66ef23d1d1eb admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.471061 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.471705 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.471978 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.472269 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.472890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.474483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.497486 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1389/2795] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:37.503164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.516580 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b3cc-ae86-48fb-9d84-a01352d123b7 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.522240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.522936 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.523266 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.523634 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.524301 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.525911 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a51929-b044-4352-ae86-23fa3dea4ec7 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.538292 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1407/2796] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:37.544168 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.554345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459f1c6-08a5-4c2e-ac83-eacc6651249d admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.559067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.559720 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.560002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.560278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] PATH_INFO: `/v3/projects/20c0293b79e6463c8d27772bece1cb9f/users/0551fb84416f491c912609178e7609da/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.575591 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c0293b79e6463c8d27772bece1cb9f, user_id=0551fb84416f491c912609178e7609da, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.578317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.597642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1390/2797] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:37 2022] PUT /identity/v3/projects/20c0293b79e6463c8d27772bece1cb9f/users/0551fb84416f491c912609178e7609da/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:37.616403 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.616839 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.617323 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.648232 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] MFA Rules not processed for user `0551fb84416f491c912609178e7609da`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:37.649826 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:37.719393 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1408/2798] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:37.731099 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.732823 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:37.767799 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:37.794141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f062cc96-a234-4d57-9c31-a2d05ce32538 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.797919 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.798669 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.798950 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.799223 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.809461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.809461 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276eebae-d272-4ab2-9546-b917400f6ce4 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.829550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1391/2799] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:37.857982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:37.920279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135871d-a784-4447-8824-b29a40cf2280 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:37.924820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:37.925439 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:37.925716 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:37.925983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] PATH_INFO: `/v3/users/0551fb84416f491c912609178e7609da` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:37.927131 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0551fb84416f491c912609178e7609da)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:37.929016 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:37.934925 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:37.975980 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0551fb84416f491c912609178e7609da'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:37.983231 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0551fb84416f491c912609178e7609da'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:37.990740 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0551fb84416f491c912609178e7609da'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:37.991699 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0551fb84416f491c912609178e7609da'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:37.992412 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0551fb84416f491c912609178e7609da'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.010804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1409/2800] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:37 2022] DELETE /identity/v3/users/0551fb84416f491c912609178e7609da => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:38.015262 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.015550 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.015827 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.034740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:38.035958 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:38.095789 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1392/2801] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:38.106445 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.108029 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:38.176091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e64bfe-b10b-4b96-abcc-420763f1cc71 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.182326 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.185120 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.193281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.193281 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.198515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.200018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19839bf8-b7bd-49fa-9675-8dd59aa95913 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.209988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1410/2802] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:38.504166 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.506437 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:38.521028 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eeb82a-9752-4763-b1e1-a88480df67ac None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.526322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.527121 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.527504 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.527900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] PATH_INFO: `/v3/projects/20c0293b79e6463c8d27772bece1cb9f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.529151 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c0293b79e6463c8d27772bece1cb9f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.531193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.580568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c0293b79e6463c8d27772bece1cb9f', 'request_id': 'req-0ab8e0c8-d680-4296-8f73-9aec59f584bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.585845 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c0293b79e6463c8d27772bece1cb9f', 'request_id': 'req-0ab8e0c8-d680-4296-8f73-9aec59f584bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.593900 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c0293b79e6463c8d27772bece1cb9f', 'request_id': 'req-0ab8e0c8-d680-4296-8f73-9aec59f584bd'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.606011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] The token cache is being invalidate because project 20c0293b79e6463c8d27772bece1cb9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:38.606299 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.606961 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0ab8e0c8-d680-4296-8f73-9aec59f584bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:38.608529 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1393/2803] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:38 2022] DELETE /identity/v3/projects/20c0293b79e6463c8d27772bece1cb9f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:38.613325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.613791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.614004 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.634330 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:38.636433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:38.705200 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1411/2804] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:38.712994 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.713276 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.713660 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.734280 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:38.735633 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:38.744211 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1394/2805] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:38.748571 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.758233 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9728120c-a177-48da-b9fc-4f3a5c04c4dd None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.763492 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.764095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.764374 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.764650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.765300 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:38.765612 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.767278 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.779663 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1412/2806] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:38.784062 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.793572 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9f42d-bfe1-4a39-abaa-0d64ea3db0ca None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.798691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.799311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.799586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.799856 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.800602 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.802519 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.831567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1395/2807] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:38.835956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.846474 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb47e3-2da9-463d-8f5d-0bfd8403512c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.861197 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:38.908458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.909060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.909335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.909603 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.910221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.912139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.934221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1413/2808] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:38 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:38.938252 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.948463 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce53516-dc5f-40ed-a69e-497383d89afe admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.953128 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.953725 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.954256 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.954538 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.955295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.956872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.967534 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1396/2809] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:38.971426 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:38.981694 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ea503-9279-4cf6-bed8-12a4f375d931 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:38.986206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:38.986834 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:38.987110 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:38.987387 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:38.988012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:38.989569 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf8ba11-556d-458a-bb6f-fad04548f269 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:38.999944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1414/2810] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:39.004118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:39.013930 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467ef2f-e3db-488a-868c-20be5c9e67c6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:39.018465 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:39.019138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:39.019416 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:39.019685 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] PATH_INFO: `/v3/projects/1aec591ef03b4642a9c5e81944ee03fd/users/b1bb3d5d586b4e9dacd9b00aa73b1385/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:39.034492 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aec591ef03b4642a9c5e81944ee03fd, user_id=b1bb3d5d586b4e9dacd9b00aa73b1385, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:39.037143 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:39.040193 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:39.054134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1397/2811] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:39 2022] PUT /identity/v3/projects/1aec591ef03b4642a9c5e81944ee03fd/users/b1bb3d5d586b4e9dacd9b00aa73b1385/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:39.065464 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:39.065753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:39.066036 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:39.093794 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] MFA Rules not processed for user `b1bb3d5d586b4e9dacd9b00aa73b1385`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:39.095088 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:39.157616 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1415/2812] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:39.168962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:39.170820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:39.227371 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dedc88-d320-4408-9fe8-40749cfc312c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:39.231706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:39.232317 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:39.232596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:39.232869 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:39.241698 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:39.243400 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1860d9-e0a3-4d87-af74-adba37f8abbb service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:39.252643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1398/2813] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:49.154883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:49.248304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=28 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:51.063269 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:51.131621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c535b-fd6d-49fb-88ad-a0800fa91a5f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:51.136035 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:51.136654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.136932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.137207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] PATH_INFO: `/v3/users/b1bb3d5d586b4e9dacd9b00aa73b1385` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.138316 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1bb3d5d586b4e9dacd9b00aa73b1385)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:51.140239 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:51.185325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1bb3d5d586b4e9dacd9b00aa73b1385'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.196049 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1bb3d5d586b4e9dacd9b00aa73b1385'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.205195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1bb3d5d586b4e9dacd9b00aa73b1385'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.205948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1bb3d5d586b4e9dacd9b00aa73b1385'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.206851 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1bb3d5d586b4e9dacd9b00aa73b1385'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.226772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1416/2814] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:51 2022] DELETE /identity/v3/users/b1bb3d5d586b4e9dacd9b00aa73b1385 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:51.232191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.232744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.232744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.261728 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:51.262999 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:51.336792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1399/2815] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:51.346739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:51.348134 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:51.408693 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d23bf-6e08-4f9f-a0fe-61ae298fcc74 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:51.412350 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:51.412997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.413226 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.413493 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.428845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:51.430289 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c21ec9-ae75-427b-9735-c34ead11a571 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:51.441294 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1417/2816] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:51.720777 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:51.722287 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:51.735054 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6b9f-e2de-498f-b4d3-789114238522 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:51.749600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:51.750629 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.753018 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.753501 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] PATH_INFO: `/v3/projects/1aec591ef03b4642a9c5e81944ee03fd` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.755844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aec591ef03b4642a9c5e81944ee03fd)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:51.759228 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:51.817138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aec591ef03b4642a9c5e81944ee03fd', 'request_id': 'req-6c44c1de-22ca-476a-a559-c786419afb0f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.823134 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aec591ef03b4642a9c5e81944ee03fd', 'request_id': 'req-6c44c1de-22ca-476a-a559-c786419afb0f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.831249 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aec591ef03b4642a9c5e81944ee03fd', 'request_id': 'req-6c44c1de-22ca-476a-a559-c786419afb0f'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.844200 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] The token cache is being invalidate because project 1aec591ef03b4642a9c5e81944ee03fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:51.844487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.845301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-6c44c1de-22ca-476a-a559-c786419afb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:51.847044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1400/2817] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:51 2022] DELETE /identity/v3/projects/1aec591ef03b4642a9c5e81944ee03fd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:51.852028 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.852314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.852586 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.873551 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:51.874905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:51.891533 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:51.936394 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1418/2818] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:51.944286 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.944570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.944846 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.965701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:51.966948 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:51.974878 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1401/2819] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:51.979167 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:51.989576 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566df5d9-6103-4773-8660-00ad5000f2a4 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:51.994136 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:51.994761 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:51.995074 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:51.995343 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:51.995991 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:51.996293 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:51.997872 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.009661 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1419/2820] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:52.014567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.024278 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5ea4c-ff7a-4f62-96c9-1c19a72e45f1 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.029717 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.030326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.030691 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.031240 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.031670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.033573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.061714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1402/2821] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:52 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:52.066018 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.076779 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da9ce0-3dd2-43c9-87c1-376ab77e259e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.137121 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.137732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.138015 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.138290 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.138961 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.140885 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.168192 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1420/2822] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:52 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:52.169395 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.179933 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f8d4b-81f1-4406-a962-d5bdb7422c17 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.184858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.185476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.185758 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.186038 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.186844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.188362 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.204339 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1403/2823] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:52.205791 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.217164 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce677e8-ae0e-4166-bf1e-e586bb5744be admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.231072 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.231692 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.232006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.232282 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.232949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.235757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1187fca-f512-455e-954e-cbc1c8ff977e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.253644 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1421/2824] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:52.254890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.266410 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa229b3a-d331-496d-8621-dfe277c33f98 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.273803 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.274493 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.274814 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.275092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] PATH_INFO: `/v3/projects/d24200bde8784623bcc84c8ea568d60d/users/6333ee5765b648319bf079547ecbffc9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.301570 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24200bde8784623bcc84c8ea568d60d, user_id=6333ee5765b648319bf079547ecbffc9, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.304588 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.318852 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1404/2825] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:52 2022] PUT /identity/v3/projects/d24200bde8784623bcc84c8ea568d60d/users/6333ee5765b648319bf079547ecbffc9/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:52.331454 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791c1c2-e32d-406c-8075-c7c85841ee6f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.331736 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791c1c2-e32d-406c-8075-c7c85841ee6f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.332006 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791c1c2-e32d-406c-8075-c7c85841ee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.360148 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8791c1c2-e32d-406c-8075-c7c85841ee6f None None] MFA Rules not processed for user `6333ee5765b648319bf079547ecbffc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:52.361356 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8791c1c2-e32d-406c-8075-c7c85841ee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:52.440059 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1422/2826] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:52.453526 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:52.455785 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:52.477956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:52.514452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f496080-9566-4711-a0db-9aedfd422529 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:52.518261 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:52.518891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:52.519165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:52.519434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:52.528184 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:52.529612 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:52.540264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1405/2827] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:55.916561 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:55.917679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:55.917679 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:55.939996 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] MFA Rules not processed for user `3de54cdce7634d7d9519722cfb67d45e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:55.941267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:55.954100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:56.001091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1423/2828] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:25:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:56.259632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:56.262701 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:56.353857 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504c03bc-4617-47b0-b9cf-86391e9e6554 service cinder] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:56.358810 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:56.359443 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:56.359751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:56.360052 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:56.362243 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:56.373564 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:56.375101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba263a8c-385a-419f-bc30-2b4d7cb8e575 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:56.384145 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1406/2829] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:05.006817 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:05.078095 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a1192-7f32-4f0f-b5d0-054da100df8b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:05.083461 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:05.084115 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:05.084395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:05.084664 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] PATH_INFO: `/v3/users/6333ee5765b648319bf079547ecbffc9` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:05.085792 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6333ee5765b648319bf079547ecbffc9)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:05.087721 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:05.141823 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6333ee5765b648319bf079547ecbffc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.156457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6333ee5765b648319bf079547ecbffc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.168513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6333ee5765b648319bf079547ecbffc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.169257 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6333ee5765b648319bf079547ecbffc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.170444 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6333ee5765b648319bf079547ecbffc9'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.195880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1424/2830] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:26:05 2022] DELETE /identity/v3/users/6333ee5765b648319bf079547ecbffc9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:05.201700 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:05.202903 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:05.203248 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:05.242872 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:05.245637 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:05.338966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1407/2831] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:05.344398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:05.345902 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:05.429338 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:05.451068 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5a771-7e99-4887-95c4-eab8a4709571 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:05.456596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:05.457417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:05.457650 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:05.458137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:05.471563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:05.472998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a36662-19b4-433b-9fcf-4ca68b24f7a7 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:05.487982 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1425/2832] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:05.824320 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:05.825838 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:05.839321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bdc9a-de9a-46bf-97d0-1c602af2a8df None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:05.854137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:05.854137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:05.854137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:05.854137 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] PATH_INFO: `/v3/projects/d24200bde8784623bcc84c8ea568d60d` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:05.855483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24200bde8784623bcc84c8ea568d60d)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:05.855483 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:05.908548 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24200bde8784623bcc84c8ea568d60d', 'request_id': 'req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.914085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24200bde8784623bcc84c8ea568d60d', 'request_id': 'req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.917749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:05.927355 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24200bde8784623bcc84c8ea568d60d', 'request_id': 'req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.948989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] The token cache is being invalidate because project d24200bde8784623bcc84c8ea568d60d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:26:05.948989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.948989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:05.948989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1408/2833] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:26:05 2022] DELETE /identity/v3/projects/d24200bde8784623bcc84c8ea568d60d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:26:06.901621 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb4ba-6727-456a-ad09-49a75e14b07a None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:06.902139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb4ba-6727-456a-ad09-49a75e14b07a None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:06.902238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ceb4ba-6727-456a-ad09-49a75e14b07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:06.925577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-14ceb4ba-6727-456a-ad09-49a75e14b07a None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:06.926836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-14ceb4ba-6727-456a-ad09-49a75e14b07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:06.985060 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1426/2834] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:07.015048 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:07.017283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:07.113264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b9a13-20ca-4574-a6c2-4de4d7e44ad6 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:07.118039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:07.118674 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:07.118966 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:07.119288 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:07.130085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:07.132563 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:07.145203 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1409/2835] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:26:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:26:07.940520 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d585d14-2722-481d-ba0f-ea7e86f8f5f5 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:07.941075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d585d14-2722-481d-ba0f-ea7e86f8f5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:07.941075 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d585d14-2722-481d-ba0f-ea7e86f8f5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:07.961305 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-3d585d14-2722-481d-ba0f-ea7e86f8f5f5 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:07.962528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3d585d14-2722-481d-ba0f-ea7e86f8f5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:07.969742 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1427/2836] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:26:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:07.982811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:07.993759 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f077ea-ed1f-4cf4-8471-04005abe3ec0 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:07.999792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:08.000405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:08.000684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:08.000954 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:08.009699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:08.011497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:08.021804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1410/2837] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:26:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:26:08.272767 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cea92-8d4e-475f-8f47-ab4890836e8d None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:08.273220 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cea92-8d4e-475f-8f47-ab4890836e8d None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:08.273547 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32cea92-8d4e-475f-8f47-ab4890836e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:08.295965 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-f32cea92-8d4e-475f-8f47-ab4890836e8d None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:08.297193 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f32cea92-8d4e-475f-8f47-ab4890836e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:08.304852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1428/2838] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:08.316939 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:08.327755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91172e4f-e3b2-43d3-ba21-40d751813d51 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:08.333318 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:08.333956 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:08.334239 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:08.334541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:08.344245 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:08.345849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:08.355394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1411/2839] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:26:08.886600 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:08.886888 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:08.887161 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:08.908267 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:08.909566 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:08.917143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1429/2840] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:26:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:08.930059 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:08.941745 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466cd1e9-ab50-400f-8f0b-13c00b0d2715 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:08.952643 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:08.954045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:08.954541 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:08.955078 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:08.968102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:08.968102 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b035d3-ecec-41df-a22f-6c95bab19786 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:08.985122 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1412/2841] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:26:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:26:11.279813 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:11.281262 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:11.352186 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea02c749-6191-4f11-9ebb-3948d483d166 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:11.356781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:11.357422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:11.357708 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:11.357979 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] PATH_INFO: `/v3/users/de84a91a0f6e4a58813a48a3bc189f9a` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:11.359179 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=de84a91a0f6e4a58813a48a3bc189f9a)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:11.361116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:11.449118 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de84a91a0f6e4a58813a48a3bc189f9a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:11.458976 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de84a91a0f6e4a58813a48a3bc189f9a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:11.470790 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de84a91a0f6e4a58813a48a3bc189f9a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:11.474541 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de84a91a0f6e4a58813a48a3bc189f9a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:11.475268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de84a91a0f6e4a58813a48a3bc189f9a'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:11.487325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:11.518516 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1430/2842] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:26:11 2022] DELETE /identity/v3/users/de84a91a0f6e4a58813a48a3bc189f9a => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:11.520549 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:11.521096 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:11.521635 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:11.550319 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:11.551817 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:11.671100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1413/2843] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:26:11.682268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:11.685113 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:11.773013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff7d44-6c6c-42f9-91eb-ba3e1f80c12a admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:11.777880 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:11.778648 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:11.778944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:11.779268 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:11.790513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:11.792555 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:11.803944 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1431/2844] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:12.034972 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:12.037070 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:12.055740 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec301d-6ac1-458f-b829-12fad11c91a8 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:12.063848 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:12.072533 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:12.073394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:12.074890 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] PATH_INFO: `/v3/projects/c56abb478c1f44508d3f22ab53b0af8c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:12.077844 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56abb478c1f44508d3f22ab53b0af8c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:12.081111 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:12.127955 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:12.139452 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56abb478c1f44508d3f22ab53b0af8c', 'request_id': 'req-2f09af5b-3635-4982-931a-4e955c44ad7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:12.145435 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56abb478c1f44508d3f22ab53b0af8c', 'request_id': 'req-2f09af5b-3635-4982-931a-4e955c44ad7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:12.154160 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56abb478c1f44508d3f22ab53b0af8c', 'request_id': 'req-2f09af5b-3635-4982-931a-4e955c44ad7c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:12.166161 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] The token cache is being invalidate because project c56abb478c1f44508d3f22ab53b0af8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:26:12.166482 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:12.167764 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:12.169321 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1414/2845] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:26:12 2022] DELETE /identity/v3/projects/c56abb478c1f44508d3f22ab53b0af8c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:21.798691 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=30,keystone:DELETE=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:22.162967 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:38.036700 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:38.038546 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:38.117549 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af023e-c1b9-40d7-8fb1-e6a4ba0ab2d5 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:38.183573 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:38.184602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:38.184889 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:38.185296 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:38.187535 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:38.201378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:38.203091 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:38.215807 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1432/2846] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:48.209643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=58 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:04.874487 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:04.875922 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:04.890918 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f09af5b-3635-4982-931a-4e955c44ad7c admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:04.895913 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:04.896498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:04.896778 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:04.897101 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:04.898925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:04.910830 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:04.912389 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:04.923173 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1415/2847] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:04.940096 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:04.941675 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:05.009190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930932a2-37e9-40b7-9155-81dfcf6afcf5 service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.013739 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.014353 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.014672 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.014947 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] PATH_INFO: `/v3/users/3de54cdce7634d7d9519722cfb67d45e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.016061 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de54cdce7634d7d9519722cfb67d45e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.017945 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.063139 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de54cdce7634d7d9519722cfb67d45e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.069948 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de54cdce7634d7d9519722cfb67d45e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.077024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de54cdce7634d7d9519722cfb67d45e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.077674 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de54cdce7634d7d9519722cfb67d45e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.078279 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-d3a16cec-8122-400e-aebd-9dea3998f3b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de54cdce7634d7d9519722cfb67d45e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.096130 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1433/2848] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:27:04 2022] DELETE /identity/v3/users/3de54cdce7634d7d9519722cfb67d45e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:05.293754 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.312345 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060e34c-331d-44e0-8e8e-4e67692e799f service placement] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.381558 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.382198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.382479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.382783 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] PATH_INFO: `/v3/projects/50beb48049264dfe8b6a75937c1041fe` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.383934 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=50beb48049264dfe8b6a75937c1041fe)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.385841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.433069 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50beb48049264dfe8b6a75937c1041fe', 'request_id': 'req-e0359499-b28e-45d1-bee6-a456b0ce52fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.439749 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50beb48049264dfe8b6a75937c1041fe', 'request_id': 'req-e0359499-b28e-45d1-bee6-a456b0ce52fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.447506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50beb48049264dfe8b6a75937c1041fe', 'request_id': 'req-e0359499-b28e-45d1-bee6-a456b0ce52fe'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.460555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] The token cache is being invalidate because project 50beb48049264dfe8b6a75937c1041fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:27:05.460849 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.461502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e0359499-b28e-45d1-bee6-a456b0ce52fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:05.463035 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1416/2849] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:27:05 2022] DELETE /identity/v3/projects/50beb48049264dfe8b6a75937c1041fe => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:05.468806 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.469089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.469358 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.493528 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:05.494764 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:05.554143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1434/2850] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:05.562388 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.562706 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.562990 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.584573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:05.585792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:05.594082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1417/2851] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:05.599206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.610003 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b363a-1c34-4db4-b41f-331cf59f1deb None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.614629 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.615230 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.615506 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.615771 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.616470 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:27:05.616766 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.618345 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.632041 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1435/2852] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:27:05.637361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.646605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9147a66-4c24-48f5-9260-a7cd8e975ed6 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.651295 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.651927 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.652207 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.652476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.653107 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.655039 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.681797 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1418/2853] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:27:05 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:27:05.686591 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.696677 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a859c4f-f31e-4981-9534-a0cc345b13ff admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.700876 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:05.758613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.759225 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.759504 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.759772 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.760404 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.762878 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.784762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1436/2854] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:27:05 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:27:05.788977 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.799702 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c3e06-9961-491e-8a8a-b9bf254b7acc admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.804399 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.805003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.805283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.805555 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.806202 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.807840 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.818906 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1419/2855] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:05.822905 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.833864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd367d98-0b20-4ced-8fc8-279a58fd981c admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.838820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.840071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.840416 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.840695 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.841355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.843030 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e8fa4-fdbb-4630-90dd-577f6a7da24f admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.854594 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1437/2856] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:05.859512 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:05.869891 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abacdd79-e7ad-47f2-a3d0-2bbdbdf506f4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:05.874925 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:05.875569 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.875804 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.876082 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] PATH_INFO: `/v3/projects/fda379bf109f49ac93d4d83276e38741/users/34127ec235ac4d4daac382754e04e03d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:05.890907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fda379bf109f49ac93d4d83276e38741, user_id=34127ec235ac4d4daac382754e04e03d, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:05.893573 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:05.907868 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1420/2857] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:27:05 2022] PUT /identity/v3/projects/fda379bf109f49ac93d4d83276e38741/users/34127ec235ac4d4daac382754e04e03d/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:05.976988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:05.977604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:05.977604 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:06.006355 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] MFA Rules not processed for user `34127ec235ac4d4daac382754e04e03d`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:06.007602 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:06.072342 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1438/2858] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:27:06.088470 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:06.090828 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:06.105656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:06.151045 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241b639-6081-49e2-ac22-8c34c57bc9b4 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:06.154811 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5882280742154859b52fa47c350a137f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:06.155942 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:06.156226 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:06.156497 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:06.165479 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:06.166957 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1f2590-a046-4dbe-aef7-ce7e79490f81 service cinder] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:06.177723 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1421/2859] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:06 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:14.338378 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:14.409901 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd935f3-24ea-4c79-a999-855e8314c86f None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:14.414425 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:14.415089 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:14.415366 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:14.415630 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] PATH_INFO: `/v3/users/34127ec235ac4d4daac382754e04e03d` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:14.416820 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34127ec235ac4d4daac382754e04e03d)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:14.418907 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:14.437335 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:14.473168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34127ec235ac4d4daac382754e04e03d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:14.482646 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34127ec235ac4d4daac382754e04e03d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:14.492346 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34127ec235ac4d4daac382754e04e03d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:14.493187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34127ec235ac4d4daac382754e04e03d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:14.493883 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34127ec235ac4d4daac382754e04e03d'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:14.516141 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1439/2860] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:27:14 2022] DELETE /identity/v3/users/34127ec235ac4d4daac382754e04e03d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:14.520953 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:14.521506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:14.521506 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:14.540841 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:14.542058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:14.614788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1422/2861] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:27:14.624654 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:14.626098 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:14.684968 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf151467-2e78-48f8-9c6b-a4f78fe1d0d0 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:14.690565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:14.690990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:14.691168 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:14.691756 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:14.702562 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:14.704033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00de976-098f-4f77-abfb-3463aff5f2c7 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:14.716022 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1440/2862] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:27:15.014218 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.015670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:15.028169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6334f7ee-dc3c-4ea2-9303-a5fb6ac1a601 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.039290 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.040298 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.040751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.041191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] PATH_INFO: `/v3/projects/fda379bf109f49ac93d4d83276e38741` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.043303 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fda379bf109f49ac93d4d83276e38741)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.045693 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.094567 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fda379bf109f49ac93d4d83276e38741', 'request_id': 'req-18445809-a070-4de8-9712-e13e1c098994'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:15.101142 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fda379bf109f49ac93d4d83276e38741', 'request_id': 'req-18445809-a070-4de8-9712-e13e1c098994'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:15.111165 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fda379bf109f49ac93d4d83276e38741', 'request_id': 'req-18445809-a070-4de8-9712-e13e1c098994'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:15.126876 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] The token cache is being invalidate because project fda379bf109f49ac93d4d83276e38741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:27:15.127210 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:15.127714 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-18445809-a070-4de8-9712-e13e1c098994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:15.129219 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1423/2863] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:27:15 2022] DELETE /identity/v3/projects/fda379bf109f49ac93d4d83276e38741 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:15.134187 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.134450 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.134762 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.154972 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:15.156238 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:15.223712 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1441/2864] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:15.231753 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.232044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.232322 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.252382 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:15.253607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:15.262604 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1424/2865] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:15.266773 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.278389 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbda6f-99ea-41f3-95f6-af3b4ab5a9d3 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.283249 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.283895 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.284182 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.284457 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] PATH_INFO: `/v3/domains` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.285116 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80491) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:27:15.285422 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.287069 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.299190 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1442/2866] 10.222.0.49 () {58 vars in 1153 bytes} [Wed Aug 10 11:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:27:15.303601 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.313358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191f98c5-f57c-4a4c-8173-375c13ff60d4 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.319476 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.320092 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.320394 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.320661 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.321301 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.323225 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.353678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1425/2867] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:27:15 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:27:15.357932 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.368202 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c44f4e-080c-4894-a96b-f886a33f9703 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.390990 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:15.438312 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.438984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.439263 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.439531 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.440196 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.442097 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.467066 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1443/2868] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:27:15 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:27:15.471349 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.482374 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde10a7d-300c-402f-bdf7-2bccf552b841 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.487812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.488578 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.488960 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.489751 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.490568 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.492340 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.498596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:15.505167 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1426/2869] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:15.508264 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.519145 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90350b-3779-47b6-bb6b-f85c45291140 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.526529 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.527203 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.527482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.527753 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.528407 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.529997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ae59e-1589-420a-b900-9199b82d4d44 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.542411 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1444/2870] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:15.546822 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.557266 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53c18-b344-4621-8999-a74a5ba9db77 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.562671 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.563344 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.563632 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.563907 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] PATH_INFO: `/v3/projects/f177eb8b84f3461a8957a8171381638a/users/432f5f0682b143f8a4e1319f2285da5e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.579982 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f177eb8b84f3461a8957a8171381638a, user_id=432f5f0682b143f8a4e1319f2285da5e, role_id=5235ad50c96b41abb9b58fbdeb2ec7a9)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.582839 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.596678 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1427/2871] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:27:15 2022] PUT /identity/v3/projects/f177eb8b84f3461a8957a8171381638a/users/432f5f0682b143f8a4e1319f2285da5e/roles/5235ad50c96b41abb9b58fbdeb2ec7a9 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:15.601437 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a564f4-c97d-4d3b-98a3-87a41a23ff19 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.601732 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a564f4-c97d-4d3b-98a3-87a41a23ff19 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.602012 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a564f4-c97d-4d3b-98a3-87a41a23ff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.621575 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-c6a564f4-c97d-4d3b-98a3-87a41a23ff19 None None] MFA Rules not processed for user `8727d28e1a63419baaa4c9f20a7bc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:15.622847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-c6a564f4-c97d-4d3b-98a3-87a41a23ff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:15.689975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1445/2872] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:15.699884 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:15.702170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:15.762115 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a829a13-4839-4322-9fe4-46950622c80a admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:15.765858 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:15.766494 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:15.766790 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:15.767071 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:15.777029 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:15.778582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:15.788596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1428/2873] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:20.275977 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.276540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.276540 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.317325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] MFA Rules not processed for user `432f5f0682b143f8a4e1319f2285da5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:20.318565 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:20.411956 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1446/2874] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:20.416699 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.427812 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b03524-8777-42ec-aa6a-08d413cf278e service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.433154 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.433806 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.434087 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.434361 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.435036 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.437109 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.470155 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1429/2875] 10.222.0.49 () {60 vars in 1153 bytes} [Wed Aug 10 11:27:20 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:27:20.474138 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.485215 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f822c-f718-4947-b96a-84da7658ce4e None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.515917 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:20.546847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.547567 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.547846 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.548123 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] PATH_INFO: `/v3/users` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.548776 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.550711 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.575777 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1447/2876] 10.222.0.49 () {60 vars in 1144 bytes} [Wed Aug 10 11:27:20 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:27:20.579802 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.591081 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32974fb-d3d5-4e38-be1b-b0d9e7100ba5 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.596008 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.596619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.596899 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.597169 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.597816 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.599516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.610502 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1430/2877] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:20.615435 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.631157 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e3a32-c298-476d-b852-b5d009d7e9ec admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.636274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.637577 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.637858 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.638207 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] PATH_INFO: `/v3/projects/02e810b106004af8a4d30dac37154e53/users/a7dd94f2fc0548e89eeae6fd1444f304/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.664501 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e810b106004af8a4d30dac37154e53, user_id=a7dd94f2fc0548e89eeae6fd1444f304, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.668828 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.684458 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1448/2878] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:27:20 2022] PUT /identity/v3/projects/02e810b106004af8a4d30dac37154e53/users/a7dd94f2fc0548e89eeae6fd1444f304/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:20.688597 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.700085 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c528b0-57c8-4698-bbf0-740628826226 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.735402 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.737152 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:20.764716 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.765342 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.765642 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.766386 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.767586 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.769639 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.780553 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1431/2879] 10.222.0.49 () {58 vars in 1122 bytes} [Wed Aug 10 11:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:27:20.784863 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:20.794791 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09aeff-15c3-4bba-908e-8b7357938995 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.800309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.800309 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] REQUEST_METHOD: `PUT` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.800623 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.800710 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] PATH_INFO: `/v3/projects/02e810b106004af8a4d30dac37154e53/users/a7dd94f2fc0548e89eeae6fd1444f304/roles/2eda6d42cc1048c1a4fd660987f7c37f` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.802596 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e810b106004af8a4d30dac37154e53, user_id=a7dd94f2fc0548e89eeae6fd1444f304, role_id=2eda6d42cc1048c1a4fd660987f7c37f)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.803579 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53216196-ee70-460a-8a5f-f1565ed73b69 admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:20.805304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e5269-0976-44e7-bf1d-d2fbe81276db admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.807390 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:20.807994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:20.808274 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:20.809331 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:20.810873 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:20.822118 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1432/2880] 10.222.0.49 () {60 vars in 1483 bytes} [Wed Aug 10 11:27:20 2022] PUT /identity/v3/projects/02e810b106004af8a4d30dac37154e53/users/a7dd94f2fc0548e89eeae6fd1444f304/roles/2eda6d42cc1048c1a4fd660987f7c37f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:27:20.888543 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:20.890033 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:20.900703 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1449/2881] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:27:27.258756 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076be9-d56b-4742-bf0a-44c8493f09aa None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:27.259590 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076be9-d56b-4742-bf0a-44c8493f09aa None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:27.260053 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21076be9-d56b-4742-bf0a-44c8493f09aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:27.292311 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-21076be9-d56b-4742-bf0a-44c8493f09aa None None] MFA Rules not processed for user `a7dd94f2fc0548e89eeae6fd1444f304`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:27.294067 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-21076be9-d56b-4742-bf0a-44c8493f09aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:27.313825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:27.376826 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1433/2882] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:27.389314 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:27.390822 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:27.457071 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:27.490781 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283491a1-0917-4b07-9cd5-5424b0ea1f2d service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:27.554451 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eacd5dcb5de7474da98c2e7f30a4e6db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:27.555101 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:27.555385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:27.555660 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:27.659126 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:27.661100 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:27.676959 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1450/2883] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:27.821358 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] REQUEST_METHOD: `POST` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:27.821760 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:27.822108 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:27.847083 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.auth.core [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80490) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:27.848481 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:27.925831 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1434/2884] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:27:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:27.977997 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:27.979491 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:28.054983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707a311-0023-4194-80e8-8d6e9fe5f6b6 service glance] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:28.060610 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:28.061227 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:28.061507 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:28.061778 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:28.072998 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:28.075240 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:28.088597 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1451/2885] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:27:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:27:28.902100 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:28.971609 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a6f2d3-1c3d-401f-9f75-4aa7aa6de017 None None] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:28.998058 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:28.999656 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:29.001080 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:29.002283 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:29.010027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:29.134516 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:29.136011 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:29.158434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1435/2886] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:33.694280 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:33.696852 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:33.772284 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=100 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:33.823512 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba02125-29f9-4cbc-a49a-738ed01334cf service swift] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:33.829904 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:33.830564 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:33.830942 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:33.831221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:33.833082 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:33.856002 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:33.856260 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:33.870780 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1452/2887] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:37.739032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:37.740782 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:37.757007 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793db0f8-88d7-477b-9feb-61deb4262195 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:37.763425 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:37.764442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:37.764442 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:37.765047 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] PATH_INFO: `/v3/users/1734d2f354a247d680f3c6fe7071632c` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:37.767033 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1734d2f354a247d680f3c6fe7071632c)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:37.769257 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:37.772405 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=100 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:37.847737 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1734d2f354a247d680f3c6fe7071632c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:37.859620 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1734d2f354a247d680f3c6fe7071632c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:37.874855 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1734d2f354a247d680f3c6fe7071632c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:37.875615 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1734d2f354a247d680f3c6fe7071632c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:37.876222 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1734d2f354a247d680f3c6fe7071632c'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:37.900575 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1436/2888] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:27:37 2022] DELETE /identity/v3/users/1734d2f354a247d680f3c6fe7071632c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:38.132596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:38.144559 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d6ac8-54f0-45c1-bccd-a32971ba33fc service placement] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:38.214384 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:38.215614 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:38.215891 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:38.216154 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] PATH_INFO: `/v3/projects/5fa6be82e0004c208037e16b0bfc23a2` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:38.217399 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa6be82e0004c208037e16b0bfc23a2)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:38.219364 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:38.290949 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa6be82e0004c208037e16b0bfc23a2', 'request_id': 'req-a00c957c-6a03-45b9-8d04-24fa81b65d68'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.297398 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa6be82e0004c208037e16b0bfc23a2', 'request_id': 'req-a00c957c-6a03-45b9-8d04-24fa81b65d68'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.308026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa6be82e0004c208037e16b0bfc23a2', 'request_id': 'req-a00c957c-6a03-45b9-8d04-24fa81b65d68'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.326189 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] The token cache is being invalidate because project 5fa6be82e0004c208037e16b0bfc23a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80491) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:27:38.326608 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.327354 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-a00c957c-6a03-45b9-8d04-24fa81b65d68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.329221 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1453/2889] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:27:38 2022] DELETE /identity/v3/projects/5fa6be82e0004c208037e16b0bfc23a2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:38.333246 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:38.335422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:38.351090 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49515be-cdd9-4dd2-85f6-67b4d3f7b5e1 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:38.426695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:38.427607 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:38.428024 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:38.428453 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] PATH_INFO: `/v3/users/8f86f16d3c1f4c1486ff45763e951de3` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:38.429829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f86f16d3c1f4c1486ff45763e951de3)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:38.431847 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:38.493304 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f86f16d3c1f4c1486ff45763e951de3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.500610 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f86f16d3c1f4c1486ff45763e951de3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.512929 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f86f16d3c1f4c1486ff45763e951de3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.514079 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f86f16d3c1f4c1486ff45763e951de3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.514901 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f86f16d3c1f4c1486ff45763e951de3'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.532600 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:38.540238 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1437/2890] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:27:38 2022] DELETE /identity/v3/users/8f86f16d3c1f4c1486ff45763e951de3 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:38.651073 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981077b-c24d-44d2-a49f-92edd6bb506c None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:38.651360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981077b-c24d-44d2-a49f-92edd6bb506c None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:38.651633 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981077b-c24d-44d2-a49f-92edd6bb506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:38.671325 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-5981077b-c24d-44d2-a49f-92edd6bb506c None None] MFA Rules not processed for user `432f5f0682b143f8a4e1319f2285da5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:38.672622 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-5981077b-c24d-44d2-a49f-92edd6bb506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:38.728111 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:38.729670 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:38.731326 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:38.736864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1454/2891] 10.222.0.49 () {58 vars in 958 bytes} [Wed Aug 10 11:27:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:27:38.819025 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947465f4-d701-4029-ae03-cb0d564f09f8 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:38.824820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:38.825779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:38.826119 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:38.826315 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] PATH_INFO: `/v3/projects/82d74d9e4d884937828d00033bcf5f63` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:38.827708 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d74d9e4d884937828d00033bcf5f63)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:38.829825 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:38.894729 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d74d9e4d884937828d00033bcf5f63', 'request_id': 'req-e5650f9e-9c6c-4212-82f1-4f659bc02602'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.901023 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d74d9e4d884937828d00033bcf5f63', 'request_id': 'req-e5650f9e-9c6c-4212-82f1-4f659bc02602'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.911186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d74d9e4d884937828d00033bcf5f63', 'request_id': 'req-e5650f9e-9c6c-4212-82f1-4f659bc02602'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.924440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] The token cache is being invalidate because project 82d74d9e4d884937828d00033bcf5f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:27:38.924440 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.925695 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:38.927381 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1438/2892] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:27:38 2022] DELETE /identity/v3/projects/82d74d9e4d884937828d00033bcf5f63 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:27:40.944563 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:40.945106 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:40.945214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:40.967026 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:40.968258 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:41.032385 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1455/2893] 10.222.0.49 () {58 vars in 1009 bytes} [Wed Aug 10 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:41.046829 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:41.048273 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:41.119138 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5650f9e-9c6c-4212-82f1-4f659bc02602 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:41.124141 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:41.124744 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:41.125020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:41.125289 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:41.140032 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:41.142474 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:41.151198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1439/2894] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:27:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:27:51.024596 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=26 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:51.147892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:56.116336 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:56.117840 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:56.188195 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dc9b65-dc9b-47ba-8d1e-8081dc6ed637 None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:56.191849 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:56.192459 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:56.192733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:56.193001 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:56.194774 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:56.266966 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:56.268410 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:56.280975 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1456/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:57.463774 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:57.474003 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c0f8d-05a3-4648-ba49-dea1fd72193c service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:57.477605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:57.478198 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:57.478477 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:57.478779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:57.480523 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:57.494659 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:57.496125 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:57.507402 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1440/2896] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:06.274587 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=54 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:07.502316 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:12.560485 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:12.569206 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfb1e5-fc52-4f8c-b7a2-0cd5b30663c0 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:12.572836 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:12.573434 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:12.573720 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:12.573988 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:12.575723 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:12.639733 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:12.640962 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:12.650893 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1457/2897] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:22.646561 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=30 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:11.591174 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:11.592712 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:11.604556 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c0c24-c462-478d-b7b9-bc790b27ca1e service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:11.608333 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c5297e9a66d4124bc4888e5a92dc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:11.609002 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:11.609331 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:11.609709 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:11.616364 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:11.627370 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:11.629027 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:11.643587 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1441/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:21.635962 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=6 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:25.727044 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:25.728360 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:25.772014 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315fb7a-a4ca-48ff-a8ae-331170487ba8 service neutron] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:25.782866 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:25.783498 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:25.783787 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:25.784076 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:25.785914 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:25.818371 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:25.820322 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6698a09-a1a5-4e83-93d4-8cbdfabab312 service nova] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:25.835482 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1458/2899] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:27.688475 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:27.707323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd60f1f-9e4d-4abb-b609-6e24bc429556 service neutron] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:27.713719 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1561e7c10ad04eeda2e31c3e31cf3c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:27.714692 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:27.715172 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:27.715605 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:27.718403 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:27.733739 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:27.735191 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:27.747063 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1442/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:34.034395 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] REQUEST_METHOD: `POST` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:34.034906 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:34.035007 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:34.054463 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.auth.core [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] MFA Rules not processed for user `292584795f3c4e10b299942d80455a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80491) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:34.055728 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:34.063433 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1459/2901] 10.222.0.49 () {58 vars in 967 bytes} [Wed Aug 10 11:30:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:34.105684 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:34.117582 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b2aff-1516-42be-ab7d-5d80a609d551 service nova] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:34.122544 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70fc784da233409a9fa518088a4ac68d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:34.123177 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] REQUEST_METHOD: `GET` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:34.123458 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:34.124264 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:34.132755 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:34.134332 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:34.143175 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1443/2902] 10.222.0.49 () {60 vars in 1384 bytes} [Wed Aug 10 11:30:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:30:41.322092 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:41.323847 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:41.410511 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccce02-fa41-4669-9b6d-aaa7c61ad64b None None] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:41.415214 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:41.415857 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:41.416143 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:41.416417 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] PATH_INFO: `/v3/users/432f5f0682b143f8a4e1319f2285da5e` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:41.417864 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] RBAC: Authorizing `identity:delete_user(user_id=432f5f0682b143f8a4e1319f2285da5e)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:41.419994 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:41.473137 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '432f5f0682b143f8a4e1319f2285da5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.480645 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '432f5f0682b143f8a4e1319f2285da5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.491625 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '432f5f0682b143f8a4e1319f2285da5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.492380 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '432f5f0682b143f8a4e1319f2285da5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.493170 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '432f5f0682b143f8a4e1319f2285da5e'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.513984 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1460/2903] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:30:41 2022] DELETE /identity/v3/users/432f5f0682b143f8a4e1319f2285da5e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:41.727186 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:41.740593 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee74dbd-8c86-4689-97b2-454014374616 service swift] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:41.818662 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:41.819323 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:41.819619 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:41.819892 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] PATH_INFO: `/v3/projects/f177eb8b84f3461a8957a8171381638a` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:41.821088 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f177eb8b84f3461a8957a8171381638a)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:41.823170 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:41.877194 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f177eb8b84f3461a8957a8171381638a', 'request_id': 'req-0f859a71-4604-4589-9ed7-93d61e19c308'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.883904 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f177eb8b84f3461a8957a8171381638a', 'request_id': 'req-0f859a71-4604-4589-9ed7-93d61e19c308'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.892627 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f177eb8b84f3461a8957a8171381638a', 'request_id': 'req-0f859a71-4604-4589-9ed7-93d61e19c308'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.904422 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] The token cache is being invalidate because project f177eb8b84f3461a8957a8171381638a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:30:41.904820 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.905606 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:41.907133 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1444/2904] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:30:41 2022] DELETE /identity/v3/projects/f177eb8b84f3461a8957a8171381638a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:41.911242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:41.913311 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:41.929382 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec51876a-5f3c-4638-9b52-aea1868c602e admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:41.991651 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:41.992456 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:41.992804 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:41.993024 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] PATH_INFO: `/v3/users/a7dd94f2fc0548e89eeae6fd1444f304` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:41.994337 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7dd94f2fc0548e89eeae6fd1444f304)` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:41.996288 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:41.997978 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:42.056838 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7dd94f2fc0548e89eeae6fd1444f304'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.064812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7dd94f2fc0548e89eeae6fd1444f304'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.073613 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7dd94f2fc0548e89eeae6fd1444f304'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.074236 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7dd94f2fc0548e89eeae6fd1444f304'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.074983 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.notifications [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7dd94f2fc0548e89eeae6fd1444f304'} {{(pid=80491) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.094845 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1461/2905] 10.222.0.49 () {58 vars in 1224 bytes} [Wed Aug 10 11:30:41 2022] DELETE /identity/v3/users/a7dd94f2fc0548e89eeae6fd1444f304 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:30:42.309788 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Authenticating user token {{(pid=80490) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:42.312044 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.fernet_utils [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80490) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:42.326498 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f859a71-4604-4589-9ed7-93d61e19c308 admin admin] Validating token access rules against request {{(pid=80490) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:42.399779 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8727d28e1a63419baaa4c9f20a7bc777', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dd03b7da5c5c4a22928c3c3d2bce3325', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80490) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:42.400428 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:42.400711 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:42.400989 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] PATH_INFO: `/v3/projects/02e810b106004af8a4d30dac37154e53` {{(pid=80490) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:42.402214 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e810b106004af8a4d30dac37154e53)` {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:42.404247 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] RBAC: Authorization granted {{(pid=80490) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:42.412471 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:42.470430 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e810b106004af8a4d30dac37154e53', 'request_id': 'req-af14feef-c78b-467f-af32-6024290e209b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.479020 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e810b106004af8a4d30dac37154e53', 'request_id': 'req-af14feef-c78b-467f-af32-6024290e209b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.488792 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e810b106004af8a4d30dac37154e53', 'request_id': 'req-af14feef-c78b-467f-af32-6024290e209b'} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.505112 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] The token cache is being invalidate because project 02e810b106004af8a4d30dac37154e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80490) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:30:42.505434 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.506306 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG keystone.notifications [None req-af14feef-c78b-467f-af32-6024290e209b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80490) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:42.507877 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: [pid: 80490|app: 0|req: 1445/2906] 10.222.0.49 () {58 vars in 1233 bytes} [Wed Aug 10 11:30:42 2022] DELETE /identity/v3/projects/02e810b106004af8a4d30dac37154e53 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:52.090765 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:52.497484 dv-cinder-852724-1-iscsi devstack@keystone.service[80490]: DEBUG dbcounter [-] [80490] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80490) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:19.886515 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Authenticating user token {{(pid=80491) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:19.887953 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:20.026119 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bacd83-88c6-47aa-baeb-ab06066bb7ba admin admin] Validating token access rules against request {{(pid=80491) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:20.083643 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f5ea0d51694d8796b40d547d9a0474', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2311045f60974b74910d615e2363e16d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80491) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:20.084242 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] REQUEST_METHOD: `GET` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:20.084522 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] SCRIPT_NAME: `/identity` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:20.084793 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80491) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:20.086513 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.fernet_utils [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80491) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:20.097757 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:20.099344 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e220c1-566b-4e14-b02e-2c4befcc4dfa service placement] RBAC: Authorization granted {{(pid=80491) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:20.109013 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: [pid: 80491|app: 0|req: 1462/2907] 10.222.0.49 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:30.104812 dv-cinder-852724-1-iscsi devstack@keystone.service[80491]: DEBUG dbcounter [-] [80491] Writing DB stats keystone:SELECT=89 {{(pid=80491) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}